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SYSTEMS FOR IDENTIFYING, DISPLAYING, MARKING, AND 
TREATING SUSPECT REGIONS OF TISSUE 



RELATED APPLICATIONS 

[0001] This application claims as priority documents U.S. Patent Application No. 10/418,902, 
filed April 18, 2003, and U.S. Provisional Patent Application No. 60/560,384, ffled April 7, 
2004. 

5 FIELD OF THE INVENTION 

[0002] This invention relates generally to systems, methods, and apparati that &cilitate 
diagnostic evaluation of a tissue sample. More particularly, in certain embodiments, the 
invention relates to the identification, display, marking, and treatment of suspect regions of a 
tissue sample. 

10 BACKGROUND OF THE INVENTION 

[0003] It is common in the field of medicine to perform visual examination to diagnose 
disease. For example, visual examination of the cervix can discern areas v^rhere there is a 
suspicion of pathology. However, direct visual observation alone may be inadequate for proper 
identification of an abnormal tissue sample, particularly in the early stages of disease. 

IS [0004] In some procedures, such as colposcopic examinations, a chemical agent, such as acetic 
acid, is applied to enhance the differences in appearance between normal and pathological tissue. 
Such acetowhitening techniques may aid a colposcopist in the determination of areas in which 
there is a suspicion of pathology. 

[0005] Colposcopic techniques are not perfect. They generally require analysis by a highly- 
20 trained physician. Colposcopic images may contain complex and confiising patterns and may be 
affected by glare, shadow, or the presence of blood or other obstruction, rendering an 
indeterminate diagnosis. 

[0006] Spectral analysis has increasingly been used to diagnose disease in tissue. Spectral 
analysis is based on the principle that the intensity of light that is transmitted firom an illuminated 
25 tissue sample may indicate the state of healflx of the tissue. As in colposcopic examination, 
spectral analysis of tissue may be conducted using a contrast agent such as acetic acid. In 
spectral analysis, the contrast agent is. used to enhance differences in the light that is transmitted 
firom nomial and pathological tissues. 
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[0007] Spectral analysis offers the prospect of at least partially-automated diagnosis of tissue 
using a classification algorithm. However, examinations using spectral analysis may be 
adversely affected by glare, shadow, or the presence of blood or other obstruction, rendering an 
indeterminate diagnosis. Some artifacts may not be detectable by analysis of the spectral data 
5 alone; hence, erroneous spectral data may be inseparable from valid spectral data. Also, the 
surface of a tissue sample under spectral examination is generally not homogeneous. Areas of 
disease may be interspersed among neighboring healthy tissue, rendering overly-dififtise spectral 
data erroneous. 

[0008] Furthermore, current methods of displaying data based on tissue classification 
10 algorithms do not facilitate quick, accurate, or clear communication of diagnostic results. 
Current techniques generally reqmre interpretation by a skilled medical professional for 
meaningful and accurate conveyance of diagnostic information, due, in part, to the unfiltered 
nature of the diagnostic data. 

[0009] Additionally, current in-vivo tissue analysis methods generally cannot provide results 
15 immediately foUowhig examination. Therefore, a follow-up examination to review suspect 
regions identified in an initial examination must generally be conducted during a separate, 
follow-up patient visit. Similarly, biopsy and/or treatment of tissue cannot be performed 
immediately following an in-vivo tissue examination. Furthermore, re-examination of tissue just 
prior to biopsy or treatment may be required to verify the location of suspect portions of the 
20 tissue that were originally identified during an initial exammation, because of normal shifting, 
re-orientation, or modification of the tissue between office visits. 

[0010] Thus, there exists a need to improve the speed, accuracy, ease, and clarity with which 
an m-vivo tissue sample is examined and diagnosed. 

[0011] Furthermore, current treatment methods for cancers, pre-cancers (such as cervical 
25 neoplasia), and other diseases are lunited in that they not only kill diseased cells, they also 
damage nearby healthy tissue. 

[0012] Photodynamic therapy (PDT) is a relatively new treatment method in which 
photoreactive drugs are selected to preferentially concentrate in tumors or other diseased areas. 
The drug molecules are activated by nonionizing optical energy directed at the tissue of interest, 
30 and the photoactivated drug molecule then typically reacts with oxygen to produce chemicals 
that kill nearby cells. While PDT is effective at killing undesired cells, it can also damage 
nearby healthy tissue, and, thus, its utility is limited. 
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[0013] Another treatment method is cryoablation, also called cryotherapy or cryosurgery. In 
this method, Hquid nitrogen or other extremely cold agent is used to freeze cells, thereby 
destroying them. However, this method destroys surrounding healthy tissue as well. 
[0014] Direct chemical treatments for cervical neoplasia and/or its viral causative agent, the 
5 Human Papillomavirus (HPV), are also in use or are m development These chemical agents can 
also destroy nearby healthy tissue. Furthermore, these chemical agents can be very expensive to 
produce, and diffiise and/or unnecessary use should be avoided. 
[0015] Traditional surgical treatments, including cold knife and loop electrical excision 
procedures, also clearly may affect healthy tissue that surrovinds diseased tissue. 
10 [0016] Thus, there is a need to better localize diseased areas of tissue to avoid damage to 
healthy surrounding tissue. 

SUMMARY OF THE INVENTION 

[0017] The invention provides systems for automatically localizing suspect areas of a tissue 
15 sample. The invention further provides systems for displaying, marking, and treating the 

identified areas so that healthy tissue suffers minimal impact upon treatment and/or excision of 
the diseased tissue. The invention facilitates mmimally-invasive treatment of diseased tissue by 
accurately directing treatment to only those areas that are identified as diseased, or in which 
there is a suspicion of pathology. 
20 [0018] Thus, the invention provides a diagnostic method that includes examining an in-vivo 
tissue sample with an optical signal detection system that automatically assigns a classification to 
each of a plurality of regions of the tissue; creating an overlay map indicating the assigned 
classifications; and displaying the overlay map to facilitate identification of suspect portions of 
the tissue sample. 

25 [0019] In one embodiment, the displaying step includes superimposing the overlay m^ onto a 
reference image of the tissue sample. Where the tissue sample includes cervical tissue, the 
reference image can be a colposcopic image. Furthermore, in one embodiment, the overlay map 
is superimposed onto a real-time colposcopic image of the tissue sample, for example, a video 
image, hi an alternative embodiment, the overlay map is projected directiy onto the tissue 

30 sample, itself. 

[0020] The diagnostic method can be performed during the course of a single patient visit. In 
one embodiment, the displaying step is performed substantially contemporaneously with the 
examining step such that diagnostic data can be viewed during and/or immediately following a 
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tissue scan. In one embodiment, the classification assigned by the optical signal detection 
system is a tissue-class probability. Altematively, the classification can be a categorical label or 
description of a given region, such as necrotic, CIN 2/3, NED (no evidence of disease), and 
indeterminate. 

S [0021] In one embodiment, the diagnostic method fiurther includes the steps of identifying at 
least one suspect portion of the tissue sample and marking the at least one suspect portion. The 
marking may be done to facilitate follow-up exanunation and/or treatment following an initial 
scan. The markmg may be performed using an endogenous agent, an exogenous agent, a 
photobleaching technique, and/or a photoactivation technique. The invention allows 

10 identification and marking of suspect portions of the tissue sample to be performed in a single 
patient visit. In one embodiment, the diagnostic method further includes identifying at least one 
suspect portion of the tissue sample and excising tissue fi-om the portion for biopsy. 
[0022] The invention also provides methods of treating suspect portions following their 
identification. Various treatment methods that are compatible with methods of the invention 

IS include, for example, photodynamic therapy, cryotherapy, and direct chemical treatment. 
Fhotodynamic therapy can involve the use of one or more photosensitive agents, including 
exogenous agents and/or endogenous agents. The photosensitive agent can include a 
hematoporphyrin, a phthalocyanine, and/or a chlorin. Examples include dihematoporphyrin, 5- 
aminolevulinic acid, protoporphyrin DC, temoporfin, and mesotetrahydroxyphenylchlorin. The 

20 photosensitive agent is generally activated by nonionizing optical energy directed at the tissue of 
interest, and the photoactivated drug molecule then typically reacts with oxygen to produce 
chemicals that kill nearby cells. Methods of the invention allow application of the photosensitive 
agent only on areas in which there is high suspicion of disease, thereby having minimal 
damaging impact on surrounding healthy tissue. 

25 [0023] The treatment step can also include removing tissue fix>m a suspect portion using laser 
ablation. Again, methods of the invention allow treatment dnected specifically within diseased 
areas, thereby limiting damage of healthy tissue. 

[0024] Furthermore, the invention allows treatment immediately following an optical scan, so 
that examination and treatment can occur during a single patient visit. 
30 [0025] In another aspect, the invention provides a system for detection of suspect portions of a 
tissue sample including an optical signal detection apparatus for obtaining one or more optical 
signals firom each of a plurality of regions of a tissue sample; a memory that stores code and a 
processor that executes the code for producing an overlay map, and a display for facilitating 
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identification of suspect portions of the tissue according to the overlay map. The processor 
executes software (1) to identify a characteristic of each of the regions based at least in part on 
the one or more optical signals from the region and (2) to define the overlay map for visually 
indicating the identified characteristics. 

5 . [0026] In a preferred embodiment, the opticd signal detection apparatus includes illxmiinat^ 
optics for illuminating each of a set of regions of the tissue sample and collecting optics for 
receiving the optical signals from flie regions. Generally, the illuminating optics include a white 
light source and an ultraviolet (UV) light source, and the one or more optical signals obtained 
from the regions mclude a fluorescence spectrum and at least one reflectance spectrum. In a 

10 preferred embodiment, a fluorescence spectrum and two reflectance spectra are obtained from 
each region. The two reflectance spectra are obtained at different angles so that one set can 
serve as a redundant set for use when the other set is affected by an artifact such as glare or an 
obstruction. 

[0027] The optical signal detection apparatus preferably includes optics for obtaining a 
15 sequence of images of tiie tissue sample for use in motion compensation. For example, optical 
data obtained from regions of the sample can be reoriented to account for slight movements 
made by the patient during a scan. 

[0028] In one embodiment, each of the plurality of regions from which optical data is obtained 
during a scan has a dimension on the order of a millimeter. For example, each region may have 

20 a diameter from about 0.1mm to Srom, or, preferably, from 0.3mm to 2nun, or more preferably, 
about 0.7mm. The optics can allow quasi-confocal analysis of the tissue sample such that 
illumination of the tissue and collection from the tissue are not diffraction limited. A scan of the 
tissue sample, particularly for a cervical scan, can be performed by the system in less than about 
30 seconds. Preferably, the scan can be performed in less than about IS seconds. 

25 [0029] The system includes a display that preferably shows the overlay map superimposed 
onto an image of the tissue, such as a real-time or stored colposcopic image of the tissue. The 
display can be viewed through a viewfinder, for example, or the display can be projected onto a 
surface, or onto the tissue itself 

[0030] The system can fiirther include one or more tools for marking a suspect region of the 
30 tissue sample, for excising suspect portions for biopsy, for treating suspect portions, and/or for 
ablating tissue within a suspect portion. The tools may be retractable components of a console 
housing the optical signal detection apparatus. In one embodiment, the system includes a light 
source for performing photodynamic therapy to treat suspect portions of the tissue sample. 
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[0031] In a further aspect, the invention is directed to an optical signal detection apparatus for 
performing a cervical scan, the apparatus including illuminating optics for sequentially 
illuminating each of a plurality of regions of a tissue sample and collecting optics for receiving at 
least one optical signal from each of tiie regions. Here, the illuminating optics include a white 

S light source and an ultraviolet source, and the optical signal includes a fluorescence spectrum 
and one or more reflectance spectra. Furthermore, the optical signal detection apparatus is 
adapted to complete a cervical scan in less than about 30 seconds, and preferably, in less than 
about 15 seconds. There are generally from about 100 to about 1000 regions of tissue scanned, 
preferably about 500 regions scanned. Moreover, the apparatus can include a light source for 

10 performing photodynamic therapy. 

[0032] In yet another aspect, the invention is directed to the use of biomarkers and optical 
probes for both diagnostic identification of neoplasias, as well as for treatment. For example, in 
one embodiment, the invention is dkected to a method for identifying a characteristic of each of 
a plurality of regions of a tissue sample, including the steps of obtaining one or more optical 

15 signals from each of a plurality of regions of a tissue sample and automatically identifying a 
characteristic of each region based at least in part on the one or more optical signals from the 
region, where the optical signals are obtained following application of an optical probe and/or a 
biological-responsive probe. Linkage of a neoplasia-specific biomarker with an optical reporter 
group can be used in combination v^th the optical detection methods of the invention to more 

20 accurately identify and localize neoplasia in vivo. Thus, the optical detection system used in 
combination v^th a fluorescent-labeled biomarker, for example, can produce an optical map of 
the cervix in which areas targeted by the biomarker are enhanced by the added fluorescent 
biomarker signal. 

25 BRIEF DESCRIPTION OF THE DRAWINGS 

[0033] The objects and features of the invention can be better understood with reference to the 
drawings described below, and the claims. The drawings are not necessarily to scale, emphasis 
instead generally being placed upon illustrating the principles of the invention. In the drawings, 
like numerals are used to indicate like parts throughout the various views. The patent or 

30 application file contains at least one drawing executed in color. Copies of this patent or patent 
application publication with color drawing(s) will be provided by the U.S. Patent and Tmdemark 
OfGce upon request and payment of the necessary fee. 
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[0034] While the invention is particularly shown and described herein with reference to 
specific examples and specific embodiments, it should be understood by those skilled in the art 
that various changes in form and detail may be made therein without departing firom the spirit 
and scope of the invention. 
S [0035] Figure 1 is a bloclc diagram featuring components of a tissue characterization system 
according to an illustrative embodiment of the invention. 

[0036] Figure 2 is a schematic representation of components of the instrument used in the 
tissue characterization system of Figure 1 to obtain spectral data and image data fi-om a tissue 
sample according to an illustrative embodiment of the invention. 
10 [0037] Figure 3 is a block diagram of the instrument used in the tissue characterization system 
of Figure 1 according to an illustrative embodiment of tiie invention. 
[0038] Figure 4 depicts a probe within a calibration port according to an illustrative 
embodiment of the invention. 

[0039] Figure 5 depicts an exemplary scan pattem used by the instrument of Figure 1 to obtain 
IS spatially-correlated spectral data and image data from a tissue sample according to an illustrative 
embodiment of the invention. 

[0040] Figure 6 depicts fi-ont views of four exemplary arrangements of illumination sources 
about a probe head according to various illustrative embodiments of the invention. 
[0041] Figure 7 depicts exemplary illumination of a region of a tissue sample using light 
20 incident to the region at two different angles according to an illustrative embodiment of the 
invention. 

[0042] Figure 8 depicts illumination of a cervical tissue sample using a probe and a speculum 
according to an illustrative ^bodiment of the invention. 

[0043] Figure 9 is a schematic representation of an accessory device for a probe marked with 
25 identifying information in the form of a bar code according to an illustrative embodiment of the 
invention. 

[0044] Figure 10 is a block diagram featuring spectral data calibration and correction 
components of the tissue characterization system of Figure 1 according to an illustrative 
embodiment of the invention. 
30 [0045] Figure 1 1 is a block diagram featuring the spectral data pre-processing component of 
ihc tissue characterization system of Figure 1 according to an illustrative embodiment of the 
invention. 
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[0046] Figure 12 shows a graph depicting reflectance spectral intensity as a function of 
wavelength using an open air target accordmg to an illustrative embodiment of the invention. 
[0047] Figure 13 shows a graph depictmg reflectance spectral intensity as a function of 
wavelength using a null target according to an illustrative embodiment of the invention. 

5 [0048] Figure 1 4 shows a graph depicting fluorescence spectral intensity as a function of 
wavelength using an open air target according to an illustrative embodiment of the invention. 
[0049] Figure 15 shows a graph depicting fluorescence spectral intensity as a function of 
wavelength using a null target according to an illustrative embodiment of the invention. 
[0050] Figure 16 is a representation of regions of a scan pattem and shows values of 

10 broadband reflectance intensity at each region using an open air target according to an 
illustrative embodiment of the invention. 

[0051] Figure 1 7 shows a graph depicting as a function of wavelength the ratio of reflectance 
spectral intensity using an open air target to the reflectance spectral intensity using a null target 
according to an illustrative embodiment of the invention. 
IS [0052] Figure 1 8 shows a graph depicting as a function of wavelength the ratio of fluorescence 
spectral intensity using an open air target to tiie fluorescence spectral intensity using a null target 
according to an illustrative embodiment of the invention. 

[0053] Figure 1 9 is a photograph of a customized target for factory/preventive maintenance 
calibration and for pre-patient calibration of the instrument used in the tissue characterization 
20 system of Figure 1 according to an illustrative embodiment of the invention. 

[0054] Figure 20 is a representation of the regions of the customized target of Figure 1 9 that 
are used to calibrate broadband reflectance spectral data according to an illustrative embodiment 
of the invention. 

[0055] Figure 21 shows a graph depicting as a function of wavelength the mean reflectivity of 
25 the 10% diffuse target of Figure 19 over the non-masked regions shown in Figure 20, measured 
using the same instrument on two different days according to an illustrative embodiment of the 
invention. 

[0056] Figure 22A shows a graph depicting, for various individual instruments, curves of 
reflectance iatensity (using the BBl li^t source), each instrument curve representing a mean of 
30 reflectance intensity values for regions confirmed as metaplasia by impression and filtered 
according to an illustrative embodiment of the invention. 
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[0057] Figure 22B shows a graph depicting, for various individual instruments, curves of 
reflectance intensity of the metaplasia-by-impression regions of Figure 22 A, after adjustment 
according to an illiistrative embodiment of the invention. 

[0058] Figure 23 shows a graph depictmg tiie spectral irradiance of a MIST traceable Quartz- 
5 Tungsten-Halogen lamp, along with a model of a blackbody emitter, used for determining an 
instrument response correction for fluorescence intensity data according to an illustrative 
embodhnent of the invention. 

[0059] Figure 24 shows a graph depicting as a function of wavelength the fluorescence 
intensity of a dye solution at each region of a 499-point scan pattern according to an illustrative 
10 embodiment of the invention. 

[0060] Figure 25 shows a graph depictmg as a function of scan position the fluorescence 
intensity of a dye solution at a wavelength corresponding to a peak intensity seen in Figure 24 
according to an illustrative embodiment of the invention. 

[0061] Figure 26 shows a graph depicting exemplary mean power spectra for various 
15 individual instruments subject to a noise performance criterion according to an illustrative 
embodiment of the invention. 

[0062] Figure 27A is a block diagram featuring steps an operator performs in relation to a 
patient scan using the system of Figure 1 according to an illustrative embodiment of the 
invention. 

20 [0063] Figure 27B is a block diagram featuring steps that the system of Figure 1 performs 
during acquisition of spectral data in a patient scan to detect and compensate for movement of 
the sample during the scan. 

[0064] Figure 28 is a block diagram showing the architecture of a video system used in the 
system of Figure 1 and how it relates to other components of the system of Figure 1 according to 
25 an illustrative embodiment of the invention. 

[0065] Figure 29 A is a single video image of a target of 10% diffuse reflectivity upon which 
an arrangement of four laser spots is projected in a target focus validation procediire according to 
an illustrative embodiment of the invention. 

[0066] Figure 29B depicts the focusmg image on the target in Figure 29A with superimposed 
30 focus rings viewed by an op^tor through a viewfinder according to an illustrative embodiment 
of the invention. 

[0067] Figure 30 is a block diagram of a target focus validation procedure according to an 
illustrative embodiment of the invention. 
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[0068] Figure 3 1 illustrates some of the steps of the target focus validation procedure of Figure 
30 as applied to the target in Figure 29A. 

[0069] Figure 32A represents the green channel of an RGB image of a cervical tissue sample, 
used in a target focus validation procedure accordmg to an illustrative embodiment of the 
5 invention. 

[0070] Figure 32B represents an unage of the final verified laser spots on the cervical tissue 
sample of Figure 32A, verified during application of the target focus validation procedure of 
Figure 30 according to an illustrative embodiment of the invention. 
[0071] Figure 33 depicts a cervbc model onto which laser spots are projected during an 
10 exemplary application of the target focus validation procedure of Figure 30, where the cervix 
model is off-center such that the upper two laser spots fall within the os region of the cervix 
model, according to an illustrative embodiment of the invention. 

[0072] Figure 34 shows a graph depicting, as a function of probe position, the mean of a 
measure of focus of each of the four laser spots projected onto the off-center cervix model of 
15 Figure 33 in the target focus validation procedure of Figure 30, according to an illustrative 
embodiment of the invention. 

[0073] Figure 35 shows a series of graphs depicting mean reflectance spectra for CIN 2/3 and 
non-CIN 2/3 tissues at a time prior to application of acetic acid, at a time corresponding to 
maximum whitening, and at a time corresponding to the latest time at which data was obtained - 
20 used in determining an optimal window for obtaining spectral data according to an illustrative 
embodiment of the invention. 

[0074] Figure 36 shows a graph depicting the reflectance discrimination function spectm 
useful for differentiating between CIN 2/3 and non-CIN 2/3 tissues, used in determining an 
optimal window for obtaining spectral data according to an iUustrative embodiment of the 
25 invention, 

[0075] Figure 37 shows a graph depicting the performance of two LDA (linear discriminant 
analysis) models as applied to reflectance data obtained at various times following application of 
acetic acid, used in determining an optimal window for obtaining spectral data according to an 
illustrative embodiment of the invention. 
30 [0076] Figure 3 8 shows a series of graphs depicting mean fluorescence spectra for CIN 2/3 and 
non-CIN 2/3 tissues at a time prior to application of acetic acid, at a time corresponding to 
maximum whitening, and at a time corresponding to the latest time at which data was obtained. 
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used in determining an optimal window for obtaining spectral data according to an illustrative 
embodiment of the invention. 

[0077] Figure 39 shows a graph depicting the fluorescence discrimination function spectra 
useful for differentiating between CIN 2/3 and non-CIN 2/3 tissues in determining an optimal 

5 window for obtaining spectral data according to an illustrative embodiment of the invention. 
[0078] Figure 40 shows a graph depicting the performance of two LDA (linear discriminant 
analysis) models as applied to fluorescence data obtained at various times following application 
of acetic acid, used in determining an optimal window for obtaining spectral data according to an 
illustrative embodiment of the inventioiL 

10 [0079] Figure 41 shows a graph depicting the performance of three LDA models as applied to 
data obtained at various times following application of acetic acid, used in determining an 
optimal window for obtaining spectral data according to an illustrative embodiment of the 
invention. 

[0080] Figure 42 shows a graph depicting the determination of an optimal time window for 
15 obtaining diagnostic optical data using an optical amplitude trigger, according to an illustrative 
embodiment of the invention. 

[0081] Figure 43 shows a graph depicting the determination of an optimal time window for 
obtaining diagnostic data usmg a rate of change of mean reflectance signal trigger, according to 
an illustrative embodiment of the invention. 

20 [0082] Figure 44A represents a 480 x 500 pixel image from a sequence of images of in vivo 
human cervix tissue and shows a 256 x 256 pixel portion of the image from which data is used in 
determining a correction for a misalignment between two images from a sequence of images of 
the tissue in the tissue characterization system of Figure 1, according to an illustrative 
embodiment of the invention. 

25 [0083] Figure 44B depicts the image represented in Figure 44A and shows a 128 x 128 pixel 
portion of the image, made up of 16 individual 32 x 32 pixel validation cells, from which data is 
used in performing a validation of the misaligimaent correction determination according to an 
illustrative embodiment of the invention. 

[0084] Figure 45 is a schematic flow diagram depicting steps in a method of determining a 
30 correction for image misalignment in the tissue characterization system of Figure 1, according to 
an illustrative embodiment of the inventioiL 
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[0085] Figures 46A and 46B show a schematic flow diagram depicting steps in a version of the 
method shown in Figure 45 of determining a correction for image misalignment according to an 
illustrative embodiment of the invention. 

[0086] Figures 47A and 47B show a schematic flow diagram depicting steps in a version of flie 
5 method shown in Figure 45 of determining a correction for image misalignment according to an 
illustrative embodiment of the invention. 

[0087] Figures 48A-F depict a subset of adjusted images from a sequence of images of a tissue 
with an overlay of gridlines showing the validation cells used in validating the determinations of 
misalignment correction between the images according to an illustrative embodiment of the 
10 invention. 

[0088] Figure 49A depicts a sample image after application of a 9-pixel size (9 x 9) Laplacian 
of Gaussian filter (LoG 9 filter) on an exemplary image from a sequence of images of tissue, 
used in determining a correction for image misalignment, according to an illustrative 
embodiment of the invention. 
15 [0089] Figure 49B depicts the appHcationofboth a feathering technique and a Laplacian of 
Gaussian filter on the exemplary image used in Figure 49A to account for border processing 
effects, used in determining a correction for image misalignment according to an illustmtive 
embodiment of the invention. 

[0090] Figure 50A depicts a sample image after application of a LoG 9 filter on an exemplary 
20 image from a sequence of images of tissue, used in determining a correction for image 
misalignment according to an iliustratiye embodiment of the invention. 
[0091] Figure SOB depicts the application of both a Hamming wmdow technique and a LoG 9 
filter on the exemplary image in Figure 50A to account for borda: processing effects in the 
determination of a correction for image misalignment according to an illustrative embodiment of 
25 the invention. 

[0092] Figures 5 1 A-F depict the determination of a correction for image misalignment using 
methods including the application of LoG filters of various sizes, as well as the application of a 
Hamming window technique and a feathering technique according to illustrative embodiments of 
the invention. 

30 [0093] Figure 52 shows a graph depicting exemplary mean values of reflectance spectral data 
as a function of wavelength for tissue regions affected by glare, tissue regions affected by 
shadow, and tissue regions affected by neither glare nor shadow according to an illustrative 
embodiment of the invention. 
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[0094] Figure 53 shows a graph depicting mean values and standard deviations of broadband 
reflectance spectral data using the BBl channel light source for regions confirmed as being 
obscured by blood, obscured by mucus, obscured by glare from the BBl source, obscured by 
glare from the BB2 source, or unobscured, according to an illustrative embodiment of the 
5 invention. 

[0095] Figure 54 shows a graph depicting mean values and standard deviations of broadband 
reflectance spectral data using the BB2 channel light source for regions confirmed as being 
obscured by blood, obscured by mucus, obscured by glare &om the BBl source, obscured by 
glare &om the BB2 source, or unobscured, according to an illustrative embodiment of the 
10 mvention. 

[0096] Figure 55 shows a graph depictmg the weighted difference between the mean 
reflectance values of glare-obscured regions and unobscured regions of tissue as a function of 
wavelength used in determining metrics for application in the arbitration step in Figure 1, 
according to an illustrative embodiment of the invention. 
IS [0097] Figure 56 shows a graph depicting tiie weighted difference between the mean 

reflectance values of blood-obscured regions and unobscured regions of tissue as a function of 
wavelength used in determining metrics for application in the arbitration step in Figure 1, 
according to an illustrative embodiment of the invention. 

[0098] Figure 57 shows a graph depicting the weighted difference between the mean 
20 reflectance values of mucus-obscured regions and unobscured regions of tissue as a function of 
wavelength, used in determining metrics for application in the arbitration step in Figure 1 
according to an illustrative embodiment of the invention. 

[0099] Figure 58 shows a graph depicting a ratio of the weighted differences between the mean 
reflectance values of glare-obscured regions and unobsciired regions of tissue at two 
25 wavelengths, used in determining metrics for application in the arbitration step in Figure 1 
according to an illustrative embodiment of the invention. 

[0100] Figure 59 shows a graph depicting a ratio of the weighted differences between the mean 
reflectance values of blood-obscured regions and xmobscured regions of tissue at two 
wavelengths, used in determining metrics for application in the arbitration step in Figure 1 
30 according to an illustrative embodiment of the invention. 
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[0101] Figure 60 shows a graph depicting a ratio of the weighted differences between the mean 
reflectance values of mucus-obscured regions and unobscured regions of tissue at two 
wavelengths, used in determining metrics for application in the arbitration step in Figure 1 
according to an illustrative embodiment of the invention. 
S [0102] Figure 6 1 shows a graph depicting as a function of wavelength mean values and 

confidence intervals of a ratio of BBl and BB2 broadband reflectance spectral values for regions 
conOnned as being eitiier glare-obscured or shadow-obscured tissue, used in deteniiiiung metrics 
for application in tiie arbitration step in Figure 1 according to an illustrative embodiment of the 
invention. 

10 [0103] Figure 62 shows a graph depicting BBl and BB2 broadband reflectance spectral data 
for a region of tissue where the BBl data is affected by glare but the BB2 data is not, according 
to an illustrative embodiment of the invention. 

[0104] Figure 63 shows a graph depicting BBl and BB2 broadband reflectance spectral data 
for a region of tissue where the BB2 data is affected by shadow but the BBl data is not, 
15 according to an illustrative embodiment of the invention. 

[0105] Figure 64 shows a graph depicting BB 1 and BB2 broadband reflectance spectral data 
for a region of tissue that is obscured by blood, according to an illustrative embodiment of the 
invention. 

[0106] Figure 65 shows a graph depicting BBl and BB2 broadband reflectance spectral data 
20 for a region of tissue that is unobscured, according to an illustrative embodiment of the 
invention. 

[0107] Figure 66 shows a graph depicting the reduction in the variability of broadband 
reflectance measurements of CIN 2/3-confirmed tissue produced by applying the metrics in the 
arbitration step 128 of Figure 1 to remove data affected by an artifact, according to an illustrative 

25 embodiment of the invention. 

[0108] Figure 67 shows a graph depicting the reduction in the variability of broadband 
reflectance measurements of tissue classified as "no evidence of disease confirmed by 
pathology" produced by applying the metrics in the arbitration step 128 of Figure 1 to remove 
data affected by an artifact, according to an illustrative embodiment of the invention. 

30 [0109] Figure 68 shows a graph depicting the reduction in the variability of broadband 
reflectance measurements of tissue classified as ''metaplasia by impression" produced by 
applying the metrics in the arbitration step 128 of Figure 1 to remove data affected by an artifact, 
according to an illustrative embodiment of the invention. 
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lOllO] Figure 69 shows a graph depicting the reduction in the variability of broadband 
reflectance measurements of tissue classified as "normal by impression" produced by applying 
flie metrics in the arbitration step 128 of Figure I to remove data affected by an artifact, 
according to an illustrative embodiment of the invention. 
5 [0111] Figure 70A depicts an exemplary image of cervical tissue divided into regions for 
which two types of reflectance spectral data and one type of fluorescence spectral data are 
obtained, according to an illustrative embodiment of the invention. 

[0112] Figure 70B is a representation of the regions depicted in Figure 70A and shows the 

categorization of each region using the metrics in the arbitration step 128 of Figure 1, according 
10 to an illustrative embodiment of the invention. 

[0113] Figure 71 A depicts an exemplary image of cervical tissue divided into regions for 

which two types of reflectance spectral data and one type of fluorescence spectral data are 

obtained, according to an illustrative embodiment of the invention. 

[0114] Figure 71B is a representation of the regions depicted in Figure 71 A and shows the 
15 categorization of each region using the metrics in the arbitration step 128 of Figure 1, according 

to an illustrative embodiment of the invention. 

[0115] Figure 72A depicts an exemplary image of cervical tissue divided into regions for 
which two types of reflectance spectral data and one type of fluorescence spectral data are 
obtained, according to an illustrative embodiment of the invention. 
20 [0116] Figure 72B is a representation of the regions depicted in Figure 72A and shows the 
categorization of each region using the metrics in the arbitration step 128 of Figure 1, according 
to an illustrative embodiment of the invention. 

[0117] Figure 73 is a block diagram depicting steps in a method of processing and combining 
spectral data and image data obtained in the tissue characterization system of Figure 1 to 
25 determine states of health of regions of a tissue sample, according to an illustrative embodiment 
of the invention. 

[0118] Figure 74 is a block diagram depictmg steps in the method of Figure 73 in further 
detail, according to an illustrative embodiment of the invention. 

[0119] Figure 75 shows a scatter plot depicting discrimination between regions of noraial 
30 squamotis tissue and CIN 2/3 tissue for known reference data, obtained by comparing 

fluorescence intensity at about 460 nm to a ratio of fluorescence intensities at about 505 nm and 
about 410 nm, used m detennming an NED spectral mask (NEDspec) according to an illustrative 
embodiment of the invention. 
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[0120] Figure 76 shows a graph depicting as a function of wavelength mean broadband 
reflectance values for known normal squamous tissue regions and known CIN 2/3 tissue regions, 
used in determining an NED spectral mask (NEDspec) according to an illustrative embodiment of 
the invention. 

5 [0121] Figure 77 shows a graph depictmg as a function of wavelength mean fluorescence 
intensity values for knoAvn squamous tissue regions and known CIN 2/3 tissue regions, used in 
determining an NED spectral mask C^Dspcc) according to an illustrative embodiment of the 
invention. 

[0122] Figure 78 shows a graph depicting values of a discrimination function using a range of 
10 numerator wavelengths and denominator wavelengths in the discrimination analysis between 
known normal squamous tissue regions and known CIN 2/3 tissue regions, used in determining 
an NED spectral mask (NEDspec) accordmg to an illustrative embodunent of the invention. 
[0123] Figure 79A depicts an exemplary reference image of cervical tissue fix)m a patient scan 
in which spectral data is used in arbitration, NED spectral masking, and statistical classification 
IS of interrogation points of the tissue sample, according to an illiistrative embodiment of the 
invention. 

[0124] Figure 79B is a representation (obgram) of the interrogation points (regions) of the 
tissue sample depicted in Figure 79A and shows points classified as "filtered" following 
arbitration, "masked" following NED spectral masking with two different sets of parameters, and 
20 "CIN 2/3" following statistical classification, according to an illustrative embodiment of the 
invention. 

[0125] Figure 79C is a representation (obgram) of the interrogation points (regions) of the 
tissue sample depicted in Figure 79A and shows points classified as *'filtered" following 
arbitration, "masked" following NED spectral masking with two different sets of parameters, and 
25 "CIN 2/3" following statistical classification, according to an illustrative embodiment of the 
invention. 

[0126] Figure 79D is a representation (obgram) of the interrogation points (regions) of the 
tissue sample depicted in Figure 79A and shows points classified as "filtered" following 
arbitration, "masked" following NED spectral masking with two different sets of parameters, and 
30 "CIN 2/3" following statistical classification, according to an illustrative embodimrat of the 
invention. 

[0127] Figure 80 shows a graph depicting fluorescence intensity as a function of wavelength 
from an interrogation point confirmed as invasive carcinoma by pathology and necrotic tissue by 
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impression, used in determining a Necrosis spectral mask according to an illustrative 
embodiment of the invention, 

[0128] Figure 8 1 shows a graph depicting broadband reflectance BB 1 and BB2 as functions of 
wavelength from an interrogation point confirmed as invasive carcinoma by pathology and 
5 necrotic tissue by impression, used in determining a Necrosis spectral mask according to an 
illustrative embodiment of the invention. 

[0129] Figure 82A depicts an exemplary reference image of cervical tissue from the scan of a 
patient confirmed as having advanced invasive cancer in which spectral data is used in 
arbitration. Necrosis spectral masking, and statistical classification of interrogation points of the 

10 tissue sample, according to an illustrative embodiment of the invention. 

[0130] Figure 82B is a representation (obgram) of the interrogation points (regions) of the 
tissue sample depicted in Figure 82A and shows points classified as "filtered" following 
arbitration, "masked" following application of the "Porphyrin" and *TAD" portions of the 
Necrosis spectral mask, and "CIN 2/3" following statistical classification, according to an 

15 illustrative embodiment of the invention. 

[0131] Figure 83 shows a graph depicting as a function of wavelength mean broadband 
reflectance values for known cervical edge regions and known CIN 2/3 tissue regions, used in a 
discrimination analysis to determine a cervical edge/vaginal wall ([CEJspcc) spectral mask 
according to an illustrative embodiment of the inventiorL 

20 [0132] Figure 84 shows a graph depicting as a function of wavelength mean fluorescence 
intensity values for known cervical edge regions and known CIN 2/3 tissue regions, used in a 
discrimination analysis to determine a cervical edge/vaginal wall ([CE]spec) spectral mask 
according to an illustrative embodiment of the inventiorL 

[0133] Figure 85 shows a graph depicting as a function of wavelength mean broadband 
25 reflectance values for known vaginal wall regions and known CIN 2/3 tissue regions, used in a 
discrimination analysis to determine a cervical edge/vaginal wall ([CE]spec) spectral mask 
according to an illustrative embodiment of the invention. 

[0134] Figure 86 shows a graph depicting as a function of wavelength mean fluorescence 
intensity values for known vaginal wall regions and known CIN 2/3 tissue regions, used in a 
30 discrimination analysis to determine a cervical edge/vaginal wall ([CE]spec) spectral mask 
according to an illustrative embodiment of the invention. 
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[0135] Figure 87A depicts an exemplary reference image of cervical tissue from a patient scan 
in which spectral data is used in arbitration and cervical edge/vaginal wall ([CE]spec) spectral 
masking, according to an illustrative embodiment of the invention. 
[01361 Figure 87B is a representation (obgram) of the interrogation points (regions) of flie 
5 tissue sample depicted in Figure 87A and shows points classified as "filtered" following 
arbitration and "masked" following cervical edge/vaginal wall ([CE]spec) spectral masking, 
according to an illustrative embodiment of the invention. 

[0137] Figure 88 shows a graph depicting as a function of wavelength mean broadband 
reflectance values for known pooling fluids regions and known CIN 2/3 tissue regions, used in a 
10 discrimination analysis to determine a fluids/mucus ([MUjspec) spectral mask according to an 
illustrative embodiment of the invention. 

[0138] Figure 89 shows a graph depicting as a function of wavelength mean fluorescence 
intensity values for known pooling fluids regions and known CIN 2/3 tissue regions, used in a 
discrimination analysis to determine a fluids/mucus ([MU]spec) spectral mask according to an 

IS illustrative embodiment of the invention. 

[0139] Figure 90 shows a graph depicting as a function of wavelength mean broadband 
reflectance values for known mucus regions and known CIN 2/3 tissue regions, used in a 
discrimination analysis to determine a fluids/mucus ([MlJ]spec) spectral mask according to an 
illustrative embodiment of the invention. 

20 [0140] Figure 91 shows a graph depicting as a function of wavelength mean fluorescence 
intensity values for known mucus regions and known CIN 2/3 tissue regions, used in a 
disraimination analysis to determine a fluids/mucus ([MU]spec) spectral made according to an 
illustrative embodiment of the inventiorL 

[0141] Figure 92A depicts an exanplary reference image of cervical tissue from a patient scan 
25 in vMch spectral data is used in arbitration and fluids/mucus ([MUjspec) spectral masking, 
according to an illustrative embodiment of tiie mvention. 

[0142] Figure 92B is a representation (obgram) of the interrogation points (regions) of the 
tissue sample depicted in Figure 92A and shows points classified as "filtered" following 
arbitration and "masked" following fluids/mucus ([MU]spec) spectral masking, according to an 
30 illustrative embodiment of the invention. 

[0143] Figure 93 depicts image masks determined from an image of a tissue sample and shows 
how the image masks are combined with respect to each spectral interrogation point (region) of 
the tissue sample, according to an illustrative embodiment of the mvention. 
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[0144] Figure 94A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a corresponding glare image mask, GlarCyid, according to 
an illustrative embodiment of the invention. 

[0145] Figure 94B represents a glare image mask, Glarevid, corresponding to the exemplary 
S image in Figure 94A, according to an illustrative embodiment of the invention. 

[0146] Figure 95 is a block diagram depicting steps in a method of determining a glare image 
mask, Glarevid, for an image of cervical tissue, according to an illustrative embodiment of the 
invention. 

[0147] Figure 96 shows a detail of a histogram used in a method of determining a glare image 
10 mask, Glarcvid, for an image of cervical tissue, according to an illustrative embodiment of the 
mvention. 

[0148] Figure 97A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a corresponding region-of-interest image mask, [ROIIvid^ 
according to an illustrative embodiment of the inventioa 

15 [0149] Figure 97B represents a region-of-interest image mask, [ROr|vid> correspondmg to the 
exemplary image in Figure 120A, according to an illustrative embodiment of the invention, 
[0150] Figure 98 is a block diagram depictmg steps in a method of determining a region-of- 
interest image mask, [ROrjvid, for an image of cervical tissue, according to an illustrative 
embodiment of the mvention. 

20 [0151] Figure 99A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a corresponding smoke tube image mask, [ST]vid, 
according to an illustrative embodiment of the invention. 

[0152] Figure 99B represents a smoke tube image mask, [ST]vid, conesponding to the 
exemplary unage in Figure 99 A, according to an illustrative embodiment of flie invention. 
25 [0153] Figure 1 GO is a block diagram depicting steps in a method of determining a smoke tube 
image mask, [Srjvid, for an image of cervical tissue, according to an illustrative embodiment of 
the invention. 

[0154] Figure 101 A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a corresponding os image mask, Osyid? according to an 
30 illustrative embodiment of the invention. 

[0155] Figure lOlB represents an os image mask, Osvid, corresponding to the exemplary image 
in Figure 101 A, according to an illustrative embodiment of the invention. 
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[0156] Figure 1 02 is a block diagram depicting steps in a method of determining an os image 
mask, Osvid, for an image of cervical tissue, according to an illustrative embodiment of the 
invention. 

[0157] Figure 103 A depicts an exemplary unage of cervical tissue obtained during a patient 
5 examination and used in determining a corresponding blood image mask, Bloodvid^ according to 
an illustrative embodiment of the invention. 

[0158] Figure 1 03B represents a blood image mask, Bloodvid, corresponding to the exemplary 
image in Figure 103 A, according to an illustrative embodiment of the invention. 
[0159] Figure 1 04 is a block diagram depicting steps in a method of determining a blood 
10 image mask, Bloodvidj for an image of cervical tissue, according to an illustrative embodiment of 
the invention. 

[0160] Figure 1 05 A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a corresponding mucus image mask, MucuSvid, according to 
an illustrative embodiment of the invention. 
IS [0161] Figure 1 05B represents a mucus image mask, MucuSvid^ corresponding to the exemplary 
reference image in Figure 105A, accordmg to an illustrative embodiment of the uivention. 
[0162] Figure 106 is a block diagram depictmg steps in a method of determining a mucus 
unage mask, Mucusvid, for an image of cervical tissue, according to an illustrative embodiment 
of the invention. 

20 [0163] Figure 1 07A depicts an exemplary reference image of cervical tissue obtained during a 
patient examination and used in determining a corresponding speculum image mask, [SPjvid? 
according to an illustrative embodiment of the invention. 

[0164] Figure 1 07B represents a speculum image mask, [SP]vid, corresponding to the 
exemplary image in Figure 107A, according to an illustrative embodiment of the mvention. 
25 [0165] Figure 1 08 is a block diagram depicting steps in a method of determimng a speculum 
image mask, [SPjvids for an image of cervical tissue, according to an illustrative embodiment of 
the invention. 

[0166] Figure 1 09A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used in determining a vaginal wall image mask, [VW]vid, according to an 
30 illustrative embodiment of the invention. 

[0167] Figure 1 09B represents the image of Figure 1 09 A overlaid with a vagmal wall image 
mask, [VW]vid, following extension, determined according to an illustrative embodiment of the 
invention. 
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[0168] Figure 1 1 0 is a block diagram depicting steps in a method of determining a vaginal wall 
image mask, [VW]vid, for an image of cervical tissue, according to an illustrative embodiment of 
the invention. 

[01691 Figure 11 1 A depicts an exemplary im^e of cervical tissue obtained during a patient 
5 examination and used in determining a corresponding fluid-and-foam image mask, [FL]vid, 
according to an illustrative embodiment of the invention. 

[01701 Figure 11 IB represents a fluid-and-foam image mask, [FL]vid, corresponding to the 
exemplary image in Figure 1 1 1 A, according to an illustrative embodiment of the mvention. 
[01711 Figure 1 12 is a block diagram depicting steps in a method of determining a fluid-and- 
10 foam image mask, [FL]vid, for an image of cervical tissue, according to an illustrative 
embodiment of the invention. 

[01721 Figures 1 1 3 A-C show graphs representing a step in a method of image visual 
enhancement in which a piecewise linear transformation of an input image produces an output 
image with enhanced image brightness and contrast, accordmg to one embodhnent of the 
15 invention. 

[01731 Figure 1 14A depicts an exemplary image of cervical tissue obtained during a patient 
examination and used as a reference (base) image in a method of disease probability display, 
according to one embodiment of the invention. 

[0174] Figure 1 14B depicts the output overlay image corresponding to the reference image in 
20 Figure 1 14 A, produced using a method of disease probability display according to one 
embodiment of the invention. 

[01751 Figure 1 1 S A represents a disease display layer produced in a method of disease 
probability display for the reference rniage in Figure 1 14A, wherein CIN 2/3 probabilities at 
interrogation points are represented by circles with intensities scaled by CIN 2/3 probability, 

25 according to one embodiment of tiie invention* 

[01761 Figure 1 15B represents the disease display layer of Figure 1 14B following filtering 
using a Hamming filter, according to one embodiment of the invention, 
[01771 Figure 1 16 represents the color transformation used to determine the disease display 
layer image in a disease probability display method, according to one embodiment of the 

30 invention. 

[01781 Figure 1 1 7A depicts an exemplary reference image of cervical tissue having necrotic 
regions, obtamed during a patient examination and used as a reference (base) image in a method 
of disease probability display, accordmg to one embodiment of the invention. 
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[0179] Figure 1 1 7B depicts the output overlay image corresponding to the reference image in 
Figure 1 17A, including necrotic regions, indeterminate regions, and CIN 2/3 regions, and 
produced using a method of disease probability display according to one embodiment of the 
invention. 

S [0180] Figure 118 depicts is a block diagram of the instrument used in the tissue 

characterization system of Figure 1, additionally fitted with a tool for markmg suspect portions 
of tissue and a tool for treating the suspect portions. 

DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENT 

10 [0181] The invention provides systems and methods for obtaining spectral data and image data 
from a tissue sample, for processing the data, and for using the data to diagnose, mark, and/or 
treat suspect regions of the tissue sample. Methods of the invention include obtaining data firom 
a tissue sample, determining which data ar6 of diagnostic value, processing the useful data to 
obtain a prediction of disease state, and displaying the results in a meaningful way. In one 

IS embodiment, spectral data and image data are obtained firom a tissue saniple and are used to 
create a diagnostic map of tiie tissue sample indicating regions in which there is a high 
probability of disease. The diagnostic map is used to mark and/or treat suspect regions of tissue. 
[0182] The systems and methods of the invention can be used to perform an examination of in 
situ and/or in-vivo tissue. The invention provides for the identification of suspect regions of 

20 tissue, which are then further analyzed, excised, and/or biopsied. The systems and methods of 
the invention can be used to perform in-situ examination of the cervical tissue of a patient in a 
surgical or non-surgical setting. Non-surgical settings include, for example, a doctor's ofSce or 
examination room. The examination may be preceded or accompanied by a routine pap smear 
and/or colposcopic examination, and may be followed-up by treatment or biopsy of suspect 

25 tissue regions. 

[0183] The tissue characterization system described in detail below can be used in creating an 
overlay map indicating tissue classification categories such as Necrotic, CIN 2/3 (cervical 
intraepithelial neoplasia grades 2 and/or 3), NED (no evidence of disease), and Indeterminate. 
For example, systems of the invention can produce an overlay image with annotations 
30 corresponding to indeterminate regions, necrotic regions, and/or regions of low-to-high 

probability of high-grade disease following a tissue scan. The annotations may be displayed as 
an overlay with a reference tissue image to provide easily-discernible tissue classification results, 
for example, indicating regions of concern for the purposes of biopsy, treatment, diagnosis. 
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and/or further (follow-up) examination. The reference image may be a real-time or 
stored/captured colposcopic image, for example. 

[0184] The system described in detail herein below allows a medical professional to conduct a 
cervical scan on a patient in about five minutes or less, excluding initial calibration of the 

5 system. Certain alternative embodiments may require 10 minutes, 20 minutes, or more. The 
overlay map can be produced immediately following a patient scan, allowing on-the-spot 
diagnostic review by a medical professional. The invention allows real-time display of 
diagnostic results and/or treatment during a single patient visit. For example, data is obtained 
during an in vivo examination of the cervical tissue of a patient, the data is used in a tissue 

10 classification scheme, and results of the tissue classification scheme are displayed either during 
or immediately following the examination. 

[0185] The invention allows performing a general initial scan, and a more detailed follow-up 
scan. For example, an mitial scan is performed over a region of tissue in order to identify one or 
more areas of interest, and then the area(s) of interest are rescanned. This can be done to Ihnit 
15 the optical exposure of the tissue and/or reduce the scan time. The rescan is generally conducted 
witii higher resolution and over a smaller region than the initial scan, once suspect regions are 
identified for follow-up examination. 

[0186] A display result (i.e. a map indicating tissue classifications and/or tissue class 
probabilities) produced by the tissue characterization system is superimposed with a colposcopic 
20 (or other) image of the tissue, for example, using a beam splitter, such as a half-silvered mirror 
or a refi:active element, in a colposcope. The display result can be combined with real-time 
and/or captured images of the tissue. 

[0187] The display result can also be projected onto the tissue itself. This may be done with 
laser ligiht of a wavelength hi the visible ninge that provides contrast with the tissue (i.e. green 
25 laser near about 550 mn). Alternatively, the display result may be projected onto the tissue xising 
a laser flying spot (i.e. with human eye integration). In further embodunents, the display result is 
projected onto the tissue using focused and/or localized light sources, for example, white light or 
LED. Combinations of these techniques may be performed as well. 

[0188] The invention includes methods for marking display results on the cervix itself, using 
30 the overlay map. This can be done concurrent with or immediately following examination using 
the tissue characterization system described below, for example, to indicate suspect regions for 
follow-up examination, excision, biopsy, or treatment. The display result may be marked onto 
the tissue using exogenous agents and/or by using endogenous agrats, such as hemoglobin. In a 
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further embodiment, the display result is marked via photobleaching with light (i.e. white light, 
xenon, and/or laser light), or via photoactivation (i.e. fluorescence). Combinations of these 
marking techniques may be implemented as well. 

[0189] The display result can be projected onto the tissue to mark areas for biopsy and/or 
S treatment For example, a selected number (i.e. one, two, three, four, or more) of areas of the 
highest probability of high-grade disease are selected (eitiier automatically, semi-automatically, 
or manually) using the display result, and these areas are marked on the cervix. 
[0190] The optical detection scan of tissue results in a spatial map that generally localizes 
diseased areas within a larger region of non-diseased tissue. The spatial map can be combined 
10 with any one of a number of treatment methods in order to optimally direct such treatment while 
minimally impacting non-diseased areas. 

[0191] One such treatment method is photodynamic therapy (PDT), in which photoreactive 
drugs are selected so that they preferentially concentrate in tumors or other diseased areas. 
These molecules are then activated by nonionizing optical energy directed at the tissue region of 

IS interest The photoactivated molecule typically reacts with oxygen to produce chemicals that kill 
the cell. While PDT is effective in killing imdesired cells, it can also damage nearby healthy 
tissues. While the basic method relies on the biochemical properties of the photoreactive agent 
to preferentially concentrate in diseased tissue, this is only achieved in part. The effectiveness of 
PDT is enhanced by application of the activating illumination to only those tissues know to 

20 harbor disease. The present invention provides methods and systems for employing the optical 
scan results, wherein a map of diseased areas is produced, to direct the PDT photoactivation. 
Thus the combination of optical disease detection methods of the invention, together with a 
directed treatment, such as with PDT, enhances the treatment while limiting undesired 
consequences. 

25 [0192] A number of photosensitizers can be used and include hematoporphyrins, 

phthalocyanines, and chlorins. Example photosensitizers include dihematoporphyrin ether 
(Photofirm), 5-aminolevulinic acid (ALA), which induces protoporphyrin IX (PpIX) in tissues, 
and Foscan (temoporfin; meso-tetrahydroxyphenylchlorin). 

[0193] Photosensitive agents are generally added exogenously; however, they may also be 
30 endogenous. For example, tissues produce endogenous PpDC, which can be enhanced by 

activation of the ALA precursor. ALA occurs naturally, but it can also be enhanced by external 
administration. Topically-applied ALA is effective in the treatment of neoplastic epithelial 
lesions, including cervical intraepithelial neoplasia. Thus, in tiie present invention, the output of 
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the optical scanning system, for example, a disease map, can be used to direct topical ALA 
application on the cervix. Furthermore, the disease map can be used to direct the light used to 
activate the photosensitive agent(s). 

[0194] Both coherent and non-coherent light sources can be used for therapeutic PDT 

5 irradiation. For example, a laser can serve as a source of coherent light source, but non-coherent 
light sources such as light emitting diodes (LEDs) or broadband white light sources (such as a 
xenon flashlamp) can also be used Such sources for PDT are most effective when the activating 
light is near an absorption band of the photosensitive agent For example, Photofnn has multiple 
absorption bands, including at about 400 nm and about 620 nm. Since optical penetration depth 

10 in tissues increases at longer wavelengths, light sources in the red portion of the visible spectrum 
can be used. One such source is a frequency-doubled Nd:YAG (Neodynium: Yttrium- 
Aluminum-Gamet) laser that produces an emission wavelength of 635 nm. Alternatively an 
argon-ion-pumped dye laser can also produce a 635 nm laser light Diode lasers at 633 nm can 
also been used. A broadband light source, such as a xenon lamp, can be used in combination 

IS with commercially available optical narrow bandpass filters to select the band of interest and 
limit optical exposure from undesired wavelengths. In all these cases, the illumination source 
can be durected either through the optical scan system optics itself, or by a parallel optical 
system, so as to selectively target those areas predetermined to be the most likely to be diseased. 
[0195] Cervical neoplasia can be treated with laser ablation. In this procedure, perfomied 

20 under local anesthesia, a high-energy laser pulse is used to ablate epithelial tissue suspected of 
being diseased. For example, a CO2 laser operating at about 10.6 microns can be used, typically 
at a power density of about or greater than 1 000 W/cm^. At this wavelength, the primary 
absorption is by water, and any tissue containing water absorbs the laser energy resulting in 
tissue vaporization. One limitation of laser ablation for treatment is that tiie spatial distribution 

25 of the lesion is not always apparent. Acetic acid may be applied to the cervix to aid (by 
acetowhitening) the identification of the spatial extent of the lesion, but this is not always 
effective. Thus, the optical detection scan system of tiie invention, fi-om which a spatial map of 
the cervical disease distribution is produced, can benefit the treatment method by directing the 
application of the laser ablation. Thus, the fiiU extent of the lesion is treated with minimal 

30 damage to healthy tissue. 

[0196] Another treatment method for cervical neoplasia is cryoabalation, also called 
cryotherapy or cryosurgery. In this method extreme cold is applied to diseased tissue to destroy 
the cells. Typically a series of fireeze-and-thaw cycles are applied to a tissue region to shatter the 
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outer cell walls, killing the cells. This method also, however, destroys surrounding tissue, and 
there is therefore a need to optimally localize diseased areas. The optical detection spatial map 
can direct such procedures so as to optimize treatment of diseased tissue while minimizing 
damage to healthy tissue. 
5 [0197] Direct chemical treatments for cervical neoplasia, or for the viral causative agent, the 
Human Papillomavirus (HPV), can also be used. These include agents that can be applied 
topicaDy to the cervix to kill diseased tissue or HPV, or to inhibit further infection. Such agents 
include trichloroacetic acid, Podophyllin, Podofilox, 5-fluorouracil, interferon-alpha, HspE7, and 
imiquimod. One limitation of such chemical agents is that they can destroy healthy tissues at the 
10 same time, and/or they are very expensive to produce and therefore must be targeted in 

application. Again, spatial localization of the areas at most risk of cervical neoplasia can durect 
the therapy most effectively, with minimal impact on healthy tissue and with minimal waste of 
chemical agent. 

[0198] Even cold knife or loop electrical excision procedures can benefit firom the spatial 
15 information provided by the optical scan result The invention enables the optimal treatment of 
diseased or infected areas while leaving the healthy tissue relatively \maffected. 
[0199] A number of neoplasia-specific biomarkers may enhance the sensitivity and specificity 
for identification of cancers and pre-cancers. For example, one biomarker is the enzyme 
telomerase, which is enhanced in neoplastic cells. Biomarkers such as telomerase can be 
20 combined with optical probes for use both in diagnostic identification of neoplasias as well as for 
treatment. Fluorescent immunochemical labeling can be applied to early cervical cancer 
detection. Another biomarker associated with cervical dysplasia is DNA methylation for which 
monoclonal antibodies exist Linkage of such neoplasia-specific biomarkers with an optical 
reporter group can then be used in combination with the optical detection as described herein to 
25 more accurately identify and localize neoplasia in vivo. Thus, the optical detection system as 
described herein can produce an optical map of the cervix in which those areas targeted by the 
biomarker are enhanced by the added fluorescent biomarker signal. 
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Broadband reflectance arbitration and low-signal masking 
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20 Combining spectral and image data 
Image enhancement 
Diagnostic display 

[0200] The Table of Contents above is provided as a general organizational guide to the 
Description of the Illustrative Embodiment. Entries m the Table do not serve to limit support for 
25 any given element of the invention to a particular section of the Description. 
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Svstem 100 overview 

[0201] The invention provides systems and methods for obtaining spectral data and image data 
from a tissue sample, for processing the data, and for using the data to diagnose the tissue 
sample. As used herein, "spectral data" from a tissue sample includes data conesponding to any 

5 wavelength ofthe electromagnetic spectrum, notjxist the visible spectrum. Where exact 

wavelengths are specified, alternate embodiments comprise using wavelengths wilhin a ± 5 nm 
range of the given value, within a ± 1 0 nm range of the given value, and within a ± 25 mn range 
ofthe given value. As used herem, "image data" from a tissue sample includes data from a 
visual representation, such as a photo, a video frame, streaming video, and/or an electronic, 

10 digital or mathematical analogue of a photo, video frame, or streaming video. As used herein, a 
"tissue sample" may comprise, for example, animal tissue, human tissue, living tissue, and/or 
dead tissue. A tissue sample may be in vivo, in situ, ex vivo, or ex situ, for example. A tissue 
sample may comprise material in the vacinity of tissue, such as non-biological materials 
including dressings, chemical agents, and/or medical instruments, for example. 

15 [0202] Embodunents of the invention include obtaining data from a tissue sample, determining 
which data are of diagnostic value, processing the useful data to obtain a prediction of disease 
state, and displaymg the results in a meaningfiil way. In one embodiment, spectral data and 
image data are obtained from a tissue sample and are used to create a diagnostic map ofthe 
tissue sample showing regions in which there is a high probability of disease. 

20 [0203] The systems and methods of the invention can be used to perform an examination of in 
situ tissue without the need for excision or biopsy. In an illustrative embodiment, the systems 
and methods are used to perform in-situ examination of the cervical tissue of a patient in a non- 
surgical settmg, such as in a doctor's office or examination room. The examination may be 
preceded or accompanied by a routine pap smear and/or colposcopic examination, and may be 

25 foUowed-up by treatment or biopsy of suspect tissue regions. 

[0204] Figure 1 depicts a block diagram featuring components of a tissue characterization 
system 100 according to an illustrative embodiment of the invention. Each component ofthe 
system 100 is discussed in more detail herein. The system includes components for acquiring 
data, processing data, calculating disease probabilities, and displaying results. 

30 [0205] In the illustrative system 100 of Figure 1, an instrument 102 obtains spectral data and 
image data from a tissue sample. The instrument 1 02 obtams spectral data from each of a 
plurality of regions of the sample during a spectroscopic scan of the tissue 104. During a scan, 
video images of the tissue are also obtained by the instrument 102. Illustratively, one or more 
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complete spectroscopic spectra are obtained for each of 500 discrete regions of a tissue sample 
during a scan lasting about 12 seconds. However, in other illustrative embodiments any niunber 
of discrete regions may be scanned and the duration of each scan may vary. Since in-situ tissue 
may shift due to mvoluntary or voluntary patient movement during a scan, video hnages are used 
5 to detect shifts of Ihe tissue, and to account for the shifts in the diagnostic analysis of the tissue. 
Preferably, a detected shift is compensated for in real time 106. For example, as described below 
in fiirflier detail, one or more components of the instrument 102 may be automatically adjusted 
during the examination of a patient while q)ectral data are obtained in order to compensate for a 
detected shift caused by patient movement Additionally or alternatively, the real-time tracker 

10 106 provides a correction for patient movement that is used to process the spectral data before 
calculating disease probabilities. In addition to using image data to track movement, the 
illustrative system 100 of Figure 1 uses image data to identify regions that are obstructed or are 
outside the areas of interest of a tissue sample 108. This feature of the system 100 of Figure 1 is 
discussed herein in more detail. 

15 [0206] The system 100 shown in Figure 1 mcludes components for performing factory tests 
and periodic preventive maintenance procedures 1 10, the results of which 1 12 are used to 
preprocess patient spectral data 1 14. In addition, reference spectral calibration data are obtained 
116 in an examination setting prior to each patient exammation, and the results 1 18 of the pre- 
patient calibration are used along with the factory and preventive maintenance results 1 12 to 

20 preprocess patient spectral data 1 14. 

[0207] The instrument 102 of Figure 1 includes a frame grabber 120 for obtainmg a video 
image of the tissue sample. A focusing method 122 is applied and video calibration is performed 
124. The corrected video data may then be used to compensate for patient movement during tiie 
spectroscopic data acquisition 104. The corrected video data is also used in image masking 108, 

25 which includes identifying obstructed regions of the tissue sample, as well as regions of tissue 
that lie outside an area of diagnostic interest. In one illustrative embodiment, during a patient 
scan, a single image is used to compute unage masks 108 and to determine a brightness and 
contrast correction 126 for displaying diagnostic results. In illustrative alternative embodiments, 
more than one image is used to create image masks and/or to detemiine a visual display 

30 correction. 

[0208] In the system of Figure 1, spectral data are acquired 104 within a predetermined period 
of time following the application of a contrast agent, such as acetic acid, to the tissue sample. 
According to the illustrative embodunent, four raw spectra are obtained for each of 
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approximately 500 regions of the tissue sample and are processed. A fluorescence spectrum, two 
broadband reflectance (backscatter) spectra, and a reference spectrum are obtained at each of the 
regions over a range fi-om about 360 nm to about 720 mn wavelength. The period of time within 
which a scan is acquired is chosen so that the accuracy of the resulting diagnosis is maximized. 
5 In one illustrative embodiment, a spectral data scan of a cervical tissue sample is performed over 
an approximately 12-second period of time within a range between about 30 seconds and about 
130 seconds following application of acetic acid to the tissue sample. 
[0209] The illustrative system 1 00 includes data processing components for identifying data 
that are potentially non-representative of the tissue sample. Preferably, potentially non- 
10 representative data are either hard-masked or soft-masked. Hard-masking of data includes 
eluninating the identified, potentially non-representative data from further consideratioa This 
results in an indeterminate diagnosis in the corresponding region. Hard masks are determined in 
components 128, 130, and 108 of the syst^ 100. Soft masking includes applymg a weighting 
function or weighting factor to flie identified, potentially non-representative data. The weigihting 
15 is taken into account during calculation of disease probability 132, and may or may not result in 
an indetemiinate diagnosis in the corresponding region. Soft masks are determined in 
component 130 of the system 100. 

[0210] Soft masking provides a means of weighting spectral data according to the likelihood 
that the data is representative of clear, unobstructed tissue in a region of interest. For example, if 

20 the system 1 00 determines there is a possibility that one kind of data firom a given region is 
affected by an obstruction, such as blood or mucus, that data is '"penalized" by attributing a 
reduced weighting to that data during calculation of disease probability 132. Another kind of 
data from the same region that is determined by the system 100 not to be affected by Ihe 
obstruction is more heavily weighted in the diagnostic step than the possibly-affected data, since 

25 the unaffected data is attributed a greater weighting in tiie calculation of disease probability 132. 
[0211] In the illustrative system 1 GO, soft masking is performed in addition to arbitration of 
two or more redxmdant data sets. Arbitration of data sets is performed in component 128. In the 
illustrative embodiment, this type of arbitration employs the following steps: obtaining two sets 
of broadband reflectance (backscatter) data from each region of the tissue sample using light 

30 incident to the region at two different angles; determining if one of tiie data sets is affected by an 
artifact such as shadow, glare, or obstruction; eliminatmg one of the redundant refl^tance data 
sets so affected; and using the other data set in the diagnosis of the tissue at the region. If both of 
the data sets are unaffected by an artifact, a mean of the two sets is used. 
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[0212] According to the illustrative embodiment, the instrument 102 obtains both video images 
and spectral data from a tissue sample. The spectral data may include fluorescence data and 
broadband reflectance (backscatter) data. The raw spectral data are processed and then used in a 
diagnostic algorithm to determme disease probability for regions of the tissue sample, 
5 According to the illustrative embodiment, both hnage data and spectral data are used to mask 
data that is potentially non-representative of unobstructed regions of interest of the tissue. In 
another illustmtive embodiment, both the image data and the spectral data are alternatively or 
additionally used in the diagnostic algorithm. 

[0213] The system 100 also includes a component 132 for determining a disease probability at 
10 each of a plurality of the approximately 500 mterrogation points usmg spectral data processed in 
the components 128 and 130 and using the image masks determined in component 108. 
Illustratively, the disease probability component 132 processes spectral data with statistical 
and/or heuristics-based (non-statistically-derived) spectral classifiers 134, incorporates image 
and/or spectral mask information 136, and assigns a probability of high grade disease, such as 
IS CIN 2+, to each examined region of the tissue sample. The classifiers use stored, accumulated 
training data firom samples of known disease state. The disease display component 138 
graphically presents regions of the tissue sample having the highest probability of high gmde 
disease by employing a color map overlay of the cervical tissue sample. The disease display 
component 138 also displays regions of the tissue that are necrotic and/or regions at which a 
20 disease probability could not be determined. 

[0214] Each of the components of the illustrative system 100 is described in more detail 
below. 

Instrument - 102 

[0215] Figure 2 is a schematic representation of components of the instrument 102 used in the 
25 tissue characterization system 100 of Figure 1 to obtam spectral data and image data firom a 

tissue sample according to an illustrative embodiment of the inventioa The instrument of Figure 
2 mcludes a console 140 connected to a probe 142 by way of a cable 144. The cable 144 carries 
electrical and optical signals between the console 140 and the probe 142. In an alternative 
embodiment, signals are transmitted between the console 140 and the probe 142 wirelessly, 
30 obviating the need for the cable 144. The probe 142 accommodates a disposable component 146 
that comes into contact with tissue and may be discarded aft^ one use. The console 140 and the 
probe 142 are mechanically connected by an articulating aim 148, which can also support the 
cable 144. The console 140 contains much of the hardware and the software of ttie system, and 
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the probe 142 contains the necessary hardware for making suitable spectroscopic observations. 
The details of the instrument 100 are further explained in conjunction with Figure 3, 
[0216] Figure 3 shows an exemplary operational block diagram 150 of an instrument 102 of 
the type depicted in Figure 2. Referring to Figures 1 and 2, in some illustrative embodhnents the 
5 instrument 102 includes features of single-beam spectrometer devices, but is adapted to include 
other features of the invention. In other illustrative embodiments, the instrument 102 is 
substantially the same as double-beam spectrometer devices, adapted to include oflier features of 
the inventioiL In still o&er illustrative embodhnents the instrument 102 employs other types of 
spectroscopic devices. In the depicted embodiment, the console 140 includes a computer 152, 

10 which executes software that controls the operation of the instrument 102. The software includes 
one or more modules recorded on machine-readable media such as magnetic disks, magnetic 
tape, CD-ROM, and semiconductor memory, for example. Preferably, the machine-readable 
medium is resident withm the computer 152. In alternative embodhnents, the machme-readable 
medium can be connected to the computer 152 by a conmiunication link. However, in 

15 altemative embodiments, one can substitute computer mstructions in die form of hardwired logic 
for software, or one can substitute firmware (i.e., computer instructions recorded on devices such 
as PROMs, EPROMS, EEPROMs, or the Uke) for software. The term machme-readable 
instructions as used herein is intended to encompass software, hardwired logic, firmware, object 
code and the like. 

20 [0217] The computer 1 52 of the instrument 1 02 is preferably a general purpose computer. The 
computer 152 can be, for example, an embedded computer, a personal computer such as a lq>top 
or desktop computer, or another type of computer, that is capable of running the software, 
issuing suitable control conomands, and recording information m real-time. The illustrative 
computer 152 includes a display 154 for reportmg information to an operator of the instrument 

25 1 02, a keyboard 1 56 for enabling the operator to enter information and conmiands, and a prints 
158 for providing a print-out, or permanent record, of measurements made by the instrument 102 
and for printing diagnostic results, for example, for inclusion in the chart of a patient. According 
to the illustrative embodiment of the invention, some commands entered at the keyboard 156 
enable a user to perform certam data processing tasks, such as selectmg a particular spectrum for 

30 analysis, rejecting a spectrum, and/or selectmg particular segments of a spectrum for 

normalization. Other commands enable a user to select the wavelength range for each particular 
segment and/or to specify both wavelength contiguous and non-contiguous segments. In one 
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illustrative embodiment, data acquisition and data processing are automated and require little or 
no user input after initializing a scan. 

[0218] The illustrative console 140 also includes an ultraviolet (UV) source 160 such as a 
nitrogen laser or a frequency-tripled Nd:YAG laser, one or more white light sources 162 such as 
5 one, two, three, four, or more Xenon flash lamps, and control electronics 1 64 for controlling the 
light sources both as to intensity and as to the tune of onset of operation and the duration of 
operation. One or more power supplies 166 are included in the illustrative console 140 to 
provide regulated power for the operation of all of the components of tiie mstrument 102. The 
illustrative console 140 of Figure 3 also includes at least one spectrometer and at least one 

10 detector (spectrometer and detector 168) suitable for use with each of the light sources. In some 
illustrative embodiments, a single spectrometer operates with both the UV light source 160 and 
the white light source(s) 162. The same detector may record both UV and white light signals. 
However, in other illustrative embodiments, different detectors are used for each light source. 
[0219] The illustrative console 140 further includes couplmg optics 170 to couple the UV 

15 illumination from the UV light source 1 60 to one or more optical fibers in the cable 144 for 
transmission to the probe 142, and coupluig optics 172 for coupling the white light illumination 
from the white light source(s) 162 to one or more optical fibers in tiie cable 144 for transmission 
to the probe 142. The spectml response of a specimen to UV illumination from the UV light 
source 160 observed by the probe 142 is carried by one or more optical fibers in the cable 144 

20 for transmission to the spectrometer and detector 1 68 in the console 140. The spectral response 
of a specimen to the white light illumination from the white light source(s) 162 observed by the 
probe 142 is carried by one or more optical fibers in the cable 144 for transmission to the 
spectrometer and detector 168 in the console 140. As shown in Figure 3, the console 140 
includes a footswitch 174 to enable an operator of the instrument 102 to signal when it is 

25 appropriate to commence a spectral scan by stepping on the switch. In this manner, the operator 
has his or her hands fi:ee to perform other tasks, for example, aligning the probe 142. 
[0220] The console 140 additionally includes a calibration port 176 into which a calibration 
target may be placed for calibrating the optical components of the instrument 102. Illustratively, 
an operator places the probe 142 in registry with the calibration port 176 and issues a command 

30 that starts the calibration operation. In illustrative calibration operation, a calibrated light source 
provides a calibration signal in the form of an illumination of known intensity over a range of 
wavelengths, and/or at a number of discrete wavelengflis. The probe 142 detects the calibration 
signal, and transmits the detected signal tiirough the optical fiber in the cable 144 to the 
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spectrometer and detector 168. A test spectral result is obtained. A calibration of the spectral 
system can be computed as the ratio of the amplitude of the known illumination at a particular 
wavelength divided by the test spectral result at the same wavelength. Calibration may include 
factory calibration 1 10, preventive maintenance calibration 110, and/or pre-patient calibration 
5 1 16, as shown in the system 100 of Figure 1 . Pre-patient calibration 116 may be perfonned to 
account for patient-to-patient variation, for example. 

[0221] Figure 4 depicts the illustrative probe 142 of Figure 2 resting within a calibration port 
176 according to an illustrative embodimait of the inventioxL Referring to Figures 2-4, the 
illustrative calibration port 176 is adjustably attached to the probe 142 or the console 140 to 
10 allow an operator to perform pre-patient calibration wifliout assemblmg detachable parts. The 
pre-patient calibration port may contain one or more pre-positioned calibration targets, such as a 
customized target 426 (see also Figure 1 9) and a null target 1 87, both described in more detail 
below. 

[0222] According to the illustrative embodiment, fectory and/or preventive maintenance 
15 calibration includes using a portable, detachable calibration port to calibrate any number of 
individual units, allowing for a standardi2Bd calibration procedure among various instrumaits. 
Preferably, the calibration port 176 is designed to prevent stray room light or other external light 
from affecting a calibration measurement when a calibration target is in place in the calibration 
port 176. For example, as shown in Figure 4, the null target 187 can be positioned up against the 
20 probe head 1 92 by way of an actuator 1 89 such that the effect of external stray light is 

minimized. When not in use, the null target 187 is positioned out of the path of light between the 
customized target 426 and the collection optics 200, as depicted in Figure 4. An additional 
fittmg may be placed over the probe head 192 to further reduce the effect of external stray light 
According to one illustrative embodiment, the target 187 in the calibration port 176 is located 
25 approximately 100 mm from the probe head 192; and the distance light travels from the target 
187 to the first optical component of the probe 142 is approximately 130 mm. The location of 
the target (m relation to the probe head 192) during calibration may approximate the location of 
tissue during a patient scan. 

[0223] The illustrative probe 1 42 includes probe optics 1 78 for illuminatmg a specimen to be 
30 analyzed with UV light from the UV source 160 and for collecting the fluorescent and broadband 
reflectance (backscatter) illumination from the specimen being analyzed. The illustrative probe 
142 of Figures 2 and 3 includes a scanner assembly 180 that provides illumination fix>m the UV 
source 1 60, for example, in a raster pattem over a target area of the specimen of cervical tissue to 
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be analyzed The probe 142 also includes a video camera 182 for observing and recording visual 
images of the specimen under analysis. The probe 142 also includes a targeting source 184 for 
determining where on the surface of the specunen to be analyzed the probe 142 is pointing. The 
probe 142 also includes white light optics 186 to deliver white light from the white light 
5 source(s) 1 62 for recording the reflectance data and to assist the operator in visualizing tiie 
specimen to be analyzed. Once the operator aligns the instrument 102 and depresses the 
footswitch 174, the computer 152 controls the actions of the light sources 160, 162, the coupling 
optics 170, 172, the transmission of light signals and electrical signals through the cable 144, the 
operation of the probe optics 178 and the scanner assembly 180, the retrieval of observed 
10 spectra, the coupling of the observed spectm mto the spectrometer and detector 168 via the cable 
144, the operation of the spectrometer and detector 168, and the subsequent signal processing 
and analysis of the recorded spectra. 

[0224] Figure 4 depicts the probe 142 having top and bottom illumination sources 188, 190 
according to an illustrative embodiment of the invention. In this embodiment, the illumination 

15 sources 1 88, 1 90 are situated at an upper and a lower location about the perimeter of a probe 
head 192 such that there is illuminating light incident to a target area at each of two different 
angles. In one embodiment, the target area is a tissue sample. Hie probe head 192 contains 
probe optics 178 for illumuiating regions of tissue and for collecting illumination reflected or 
otherwise emitted from regions of tissue. Illustratively, the probe optics for collecting the 

20 illumination 200 are located between the top and bottom illumination sources 188, 190. In other 
illustrative embodiments, other arrangements of the illuminating and collecting probe optics 178 
are used that allow the illumination of a given region of tissue with light incident to the region at 
more than one angle. One such arrangement includes the collectmg optics 200 positioned around 
the illuminating optics. 

25 [0225] In one illustrative embodunent, the top and bottom illumination sources 1 88, 1 90 are 
alternately tumed on and off in order to sequentially illuminate the tissue at equal and opposite 
angles relative to the collection axis. For example, the top illumination source 188 is tumed on 
while the bottom illumination source 190 is tumed off, such that spectral measurements may be 
obtained for light reflected from a region of the tissue sample 194 illuminated with light incident 

30 to the region at a first angle. This angle is relative to the surface of the tissue sample at a point 
on the region, for example. Then, the top illumination source 188 is tumed off while the bottom 
illumination source 190 is tumed on, such that spectral measurements may be obtained using 
light incident to the region at a second angle. If data obtained using one of the illumination 
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sources is adversely affected by an artifact, such as glare or shadow, then data obtained using 
another illumination source, with light incident to the region at a different angle, may be 
unaffected by the artifact and may still be useful. The spectral measurements can include 
reflectance and/or fluorescence data obtained over a range of wavelengths, 
5 [02261 According to the various illustrative embodiments, the top and the bottom illumination 
sources 1 88, 1 90 may be alternately cycled on and off more than once while obtaining data for a 
given region. Also, cycles of the illumination sources 188, 190 may overlap, such that more than 
one illumination source is on at one time for at least part of the illumination collection procedure. 
Other illumination alternation schemes are possible, depending at least in part on the 

10 arrangement of illumination sources 1 88, 1 90 in relation to the probe head 1 92. 

[0227] After data are obtained from one region of the tissue using light incident to the region at 
more than one angle, data may likewise be obtained from another region of the tissue. In the 
illustrative embodiment of Figure 4, the scaimer assembly 180 illuminates a target area of the 
tissue sample region-by-region. Illustratively, a first region is illuminated usmg light mcident to 

15 the region at more than one angle as described above, then the probe optics 178 are automatically 
adjusted to repeat the illumination sequence at a different region within the target area of the 
tissue sample. The illustrative process is repeated until a desired subset of the target area has 
been scanned. As mentioned above, preferably about five hundred regions are scanned within a 
target area having a diameter of about 25-mm. Using the instrument 102, the scan of the 

20 aforementioned five hundred regions takes about 12 seconds. In other illustrative embodiments, 
the number of regions scanned, the size of the target area, and/or the duration of the scan vary 
from the above. 

[0228] Figure 5 depicts an exemplary scan pattern 202 used by the mstrument 102 to obtain 
spatially-correlated spectral data and image data from a tissue sample according to an illustrative 

25 embodiment of the invention. Illustratively, spectral data are obtained at 499 regions of the 
tissue sample, plus one region out of the field of view of the cervix obtained, for example, for 
calibration purposes. The exemplary scan pattern 202 of Figure 5 includes 499 regions 204 
whose centers are inside a circle 206 that measures about 25.8 mm in diameter. The center of 
each region is about 1 . 1 mm away from each of the nearest surrounding regions. This may be 

30 achieved by offsetting each scan line by about 0.9527 mm in the y-direction and by staggering 
each scan line in the x-direction by about 0.55 mm. Each of the 499 regions is about 0.7 mm in 
diameter. In other illustrative embodiments, other geometries are used. 
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[0229] According to the illustrative embodiment, the spectral data acquisition component 104 
of the system 100 depicted in Figure 1 is performed using the scan pattern 202 shown in Figure 
5. A fluorescence spectrum, two broadband reflectance spectra, and a reference spectrum are 
obtained at each region 204. The two broadband reflectance spectra use light incident to the 

5 sample at two different angles. A scan preferably begins at the center region 208, which 

corresponds to a pixel in a 500 x 480 pixel video image of the tissue sample at location 250, 240. 
As discussed in more detail below, a sequence of video images of the tissue sample may be taken 
during a scan of the 499 regions shown in Figure 5 and may be used to detect and compensate 
for movement of the tissue sample during the scan. The real-time tracker component 106 of the 

10 system 100 shown in Figure 1 performs this motion detection and compensation fimction. 

Preferably, the scanner assembly 180 of Figure 3 includes controls for keeping track of the data 
obtained, detecting a stalled scan process, aborting the scan if the tissue is exposed to 
temperature or light outside of acceptable ranges, and/or monitoring and reporting errors 
detected by the spectral data acquisition component 104 of the system of Figure 1. 

15 [0230] Figure 6 depicts front views of four exemplary arrangements 210, 212, 214, 216 of 
illumination sources about a probe head 192 according to various illustrative embodiments of the 
invention. The drawings are not to scale; they serve to illustrate exemplary relative 
arrangements of illumination sources about the perimeter of a probe head 192. Other 
arrangements include positioning collecting optics 200 around the perimeter of the probe head 

20 192, about the illumination sources, or in any other suitable location relative to the illumination 
sources. The first arrangement 210 of Figure 6 has one top illumination source 218 and one 
bottom illumination source 220, which are alternately cycled on and off as described above. The 
illumination sources are arranged about the collecting optics 200, which are located in the center 
of the probe head 192. Light from an illumination source is reflected from the tissue and 

25 captured by the collecting optics 200. 

[0231] The second arrangement 212 of Figure 6 is similar to the first arrangement 210, except 
that there are two illumination sources 222, 224 in the top half of the probe head 192 and two 
illumination sources 226, 228 in the bottom half of the probe head 192. In one embodiment, the 
two lights above the midline 230 are tumed on and the two lights below the midline 230 are 

30 tumed off while obtaining a first set of spectral data; then the lights above the midline 230 are 
tumed off and the lights below the midline 230 are tumed on while obtaining a second set of 
spectral data. In an alternate illustrative embodiment, only one of the four illumination sources 
are tumed on at a time to obtain four sets of spectral data for a given region. Other illustrative 
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embodiments include turning the illumination sources on and off in other patterns. Other 
alternative embodiments include using noncircular or otherwise differently shaped illumination 
sources, and/or using a different number of illumination sources. 

[0232] The third arrangement 214 of Figure 6 includes each illumination source 232, 234 
5 positioned on either side of the probe head 192. The sources 232, 234 may be alternated in a 

manner analogous to those described for the first arrangement 210. 

[0233] The fourth arrangement 216 of Figure 6 is similar to the second arrangement 212, 

except that the illumination sources 236, 238 on the right side of the probe head 192 are turned 

off and on together, alternately with the illumination sources 240, 242 on the left side of the 
10 probe head 192. Thus, two sets of spectral data may be obtained for a given region, one set 

using the illumination sources 236, 238 on the right of ttie midline 244, and the other set using 

the illumination sources 240, 242 on the left of the midline 244. 

[0234] Figure 7 depicts exemplary illumination of a region 250 of a tissue sample 194 using 
light mcident to the region 250 at two different angles 252, 254 according to an illustrative 

IS embodiment of the invention. Figure 7 demonstrates that source light position may affect 

whether data is affected by glare. The probe head 192 of Figure 7 is depicted in a cut-away view 
for illustrative purposes. In this illustrative embodiment, the top illumination source 188 and 
bottom illumination source 190 are turned on sequentially and illuminate the surface of a tissue 
sample 194 at equal and opposite angles relative to the collection axis 256. Arrows represent the 

20 light emitted 252 from the top illumination source 188, and the light specularly reflected 258 
from the surface of the region 250 of the tissue sample 194. In preferred embodiments, it is 
desired to collect diffusely reflected light, as opposed to specularly reflected li^t 258 (glare). 
Since the specularly reflected light 258 from the top illumination source 188 does not enter the 
collecting optics 200 in the example illustrated in Figure 7, a set of data obtained using the top 

25 illumination source 1 88 would not be affected by glare. 

[0235] However, in the example illustrated in Figure 7, the emitted light 254 from the bottom 
illumination source 190 reaches the surface of the region 250 of the tissue 194 and is specularly 
reflected into the collecting optics 200, shown by the arrow 260. Data obtained using the bottom 
illumination source 190 in the example pictured in Figure 7 would be affected by glare. This 

30 data may not be useful, for example, in determining a characteristic or a condition of the region 
250 of the tissue 194. In this example, it would be advantageous to instead use the set of data 
obtained using the top illumination source 188 since it is not affected by glare. 
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[0236] The position of the collection optics 200 may affect whether or not data is affected by 
glare. For example, light 252 with illumination mtensity Io(X) strikes a tissue surface at a given 
region 250. A fraction of the initial illumination intensity, aIo(X,), is specularly reflected from the 
surface 258, where a is a real number between 0 and 1 . An acceptance cone 268 is the space 
through v^^ch light is diffusely reflected from the tissue 1 94 into the collecting optics 200, in 
this embodiment. Light may also be emitted or oflierwise transmitted from the surfece of the 
tissue. The difiEusely reflected light is of interest, since spectral data obtained from diffusely 
reflected light can be used to determine the condition of the region of the sample. If there is no 
specular reflection within the acceptance cone 268, only diffusely reflected light is collected, and 
the collected signal corresponds to lt{X), where It(A,) is the intensity of light diffusely reflected 
from the region 250 on the surface of the tissue, 

[0237] If the collection optics 200 are off-center, light incident to the tissue surfece may 
specularly reflect within the acceptance cone 268. For example, ligiht with illumination intensity 
loQ.) strikes the surface of the tissue. Light with a fraction of tiie initial illumination intensity, 
aIo(X), from a given source is specularly reflected from the surfece 266, \fbste a is a real number 
between 0 and 1. Where there is specular reflection of light witiiin the acceptance cone 268, 
both diffusely reflected light and specularly reflected light reach the collecting optics 200. Thus, 
the collected signal corresponds to an intensity represented by the sum l^X) + aIo(X). It may be 
difficult or impossible to separate the two components of the measured intensity, thus, the data 
may not be helpftd in determining the condition of the region of the tissue sample due to the 
glare effect 

[0238] Figure 8 is a diagram 284 depicting illumination of a region 250 of a cervical tissue 
sample 194 usmg a probe 142 and a vagmal speculum 286 according to an illustrative 
embodiment of the invention. Here, the illuminating light incident to tiie tissue sample 194, is 
depicted by the upper and lower intersecting cones 196, 198. In a preferred embodiment, the 
probe 142 operates without physically contacting the tissue bemg analyzed. In one embodiment, 
a disposable sheath 146 is used to cover the probe head 192, for example, m case of incidental 
contact of the probe head 192 with the patient's body. Figure 9 is a schematic representation of 
an accessory device 290 that forms at least part of the disposable sheatii 146 for a probe head 
192 according to an illustrative embodiment of the invention. In one illushative embodiment, 
the entire sheatii 146, including Hbe accessory device 290, if present is disposed of after a single 
use on a patient. As i^own in Figure 8, in one illustrative embodimenl^ the disposable sheatii 
146 and/or tiie accessory device 290 have a unique identifier, such as a two-dimensional bar 
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code 292. According to an illustrative feature, the accessory device 290 is configured to provide 
an optimal light path between the optical probe 142 and the target tissue 194. Optional optical 
elements in the accessory device 290 may be used to enhance the light transmitting and light 
receiving functions of the probe 142. 
5 [0239] Although an illustrative embodiment of the invention is described herein with req)ect 
to analysis of vaginal tissue, other tissue types may be analyzed using these methods, including, 
for example, colorectal, gastroesophageal, urinary bladder, lung, skin tissue, and/or any tissue 
comprising epithelial cells. 

Spectral calibration - 110, 112. 116 

10 [0240] Figure 10 is a block diagram 300 featuring components of the tissue characterization 
system 100 of Figure 1 that involve spectral data calibration and correction, according to an 
illustrative embodiment of the invention. The instrument 102 of Figure 1 is calibrated at the 
factory, prior to field use, and may also be calibrated at regular intervals via routine preventive 
maintenance ^M). This is referred to as factory and/or preventive maintenance calibration 1 10. 

15 Additionally, calibration is performed immediately prior to each patient scan to account for 
temporal and/or intra-patient sources of variability. This is referred to as pre-patient calibration 
1 16. The illustrative embodiment includes calibrating one or more elements of the instrument 
102, such as the spectrometer and detector 168 depicted in Figure 3. 

[0241] Calibration includes performing tests to adjust individual instrument response and/or to 
20 provide corrections accounting for individual instrument variability and/or individual test 

(temporal) variability. During calibration procedures, data is obtained for the pre-processing of 
raw spectral data from a patient scan. The tissue classification system 100 of Figure 1 includes 
determining corrections based on the factory and/or preventive maintenance calibration tests, 
indicated by block 1 12 in Figure 10 and in Figure 1 . Where multiple sets of factory and/or 
25 preventive maintenance (PM) data exists, the most recent set of data is generally used to 

determine correction factors and to pre-process spectral data fi:om a patient scan. Corrections are 
also determined based on pre-patient calibration tests, indicated by block 1 18 of Figure 10. The 
correction factors are used, at least indirectly, in the pre-processing (1 14, Figure 1) of 
fluorescence and reflectance spectral data obtained using a UV light source and two white li^t 
30 sources. Block 114 of Figure 1 1 corresponds to flie pre-processing of spectral data in the overall 
tissue classification system 100 of Figure 1, and is further discussed herein. 
[0242] Calibration accounts for sources of individual instrument variability and individual test 
variability in the preprocessing of raw spectral data from a patient scan. Sources of instrument 
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and individual test variability include, for example, external light (light originating outside the 
instrument 102, such as room light) and intemal stray light Internal stray light is due at least in 
part to intemal "cross talk," or interaction between transmitted light and the collection optics 
200. Calibration also accounts for the electronic background signal read by the instrument 102 

S when no light sources, intemal or external, are in use. Additionally, calibration accounts for 
variations in the amount of light energy delivered to a tissue sample during a scan, spatial 
inhomogeneities of the illuminaddn source(s), chromatic aberration due to die scanning optics, 
variation in the wavelength response of the collection optics 200, and/or the efSciency of the 
collection optics 200, for example, as well as other effects. 

10 [0243] In the illustrative embodiment of Figure 10, factory and preventive maintenance 

calibration tests are performed to determine correction factors 1 12 to apply to raw fluorescence 
and reflectance spectral data obtained during patient scans. The factory/preventive maintenance 
calibration tests 1 10 include a wavelength calibration test 302, a "null" target test 304, a 
fluorescent dye cuvette test 306, a tungsten source test 308, an "open air" target test 3 10, a 

IS customized target test 312, and aNIST standard target test 314. 

[0244] The wavelength calibration test 302 uses mercury and argon spectra to convert a CCD 
pixel index to wavelengths (nm). A wavelength calibration and interpolation method using data 
from the mercxjry and argon calibration test 302 is described below. 
[0245] The null target test 304 employs a target having about 0% diffuse reflectivity and is 

20 used along with other test results to accoxmt for intemal stray light. Data from the factory/PM 
null target test 304 are used to determine the three correction factors shown in block 3 16 for 
fluorescence spectral measurements (F) obtained using a UV light source, and broadband 
reflectance measurements (BBl , BB2) obtained using each of two white light sources. In one 
embodiment, these three correction factors 316 are used in determining correction factors for 

25 other tests, includmg the factory/PM fluorescent dye cuvette test 306, the factory/PM open air 
target test 3 10, the factory/PM customized target test 3 12, and the factory/PM NIST standard 
target test 3 14. The open air target test 3 10, the customized target test 3 12, and the NIST 
standard target test 3 14 are used along with the null target test 304 to correct for mtemal stray 
light in spectral measurements obtained using a UV light source and one or more white li^t 

30 sources. 

[0246] The open air target test 310 is performed without a target and in the absence of extemal 
light (all room lights turned off). The customized target test 312 employs a custom-designed 
target including a material of approximately 1 0% difiuse reflectivity and is performed in the 
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absence of external light. The custom-designed target also contains phosphorescent and 
fluorescent plugs that are used during instrument focusing and target focus validation 122. In 
one embodunent, the custom-designed target is also used during pre-patient calibration testing 
(116, 330) to monitor tiie stability of fluorescence readings between preventive maintenance 

5 procedures and/or to align an ultraviolet (UV) light source 160 - for example, a nitrogen laser or 
a frequency-tripled Nd: YAG laser. The NIST (U.S. National Institute of Standards and 
Technology) standard target test 314 employs a NIST-standard target comprismg a material of 
approximately 60% diffuse reflectivity and is performed in the absence of extemal light. 
Correction factors determined from the "open air" target test 3 10, the custom target test 3 12, and 

10 the NIST-standard target test 3 14 are shown m blocks 322, 324, and 326 of Figure 10, 
respectively. The correction factors are discussed in more detail below. 
[0247] The fluorescent dye cuvette test 306 accounts for the efSciency of the collection optics 
200 of a given unit. The illustrative embodiment uses data from the fluorescent dye cuvette test 
306 to determine a scalar correction factor 318 for fluorescence measurements (F) obtained using 

15 a UV light source. The tungsten source test 308 uses a quartz-tungsten-halogen lamp to accoimt 
for the wavelength response of the fluorescence collection optics 200, and data from this test are 
used to determine a correction factor 320 for fluorescence measurements (F) obtained using a 
UV light source. 

[0248] In addition to factory and preventive maintenance calibration 110, pre-patient 
20 calibration 1 1 6 is performed immediately before each patient scan. The pre-patient calibration 
1 1 6 includes performing a null target test 328 and a customized target test 330 before each 
patient scan. These tests are similar to the factory/PM null target test 304 and tiie factory/PM 
custom target test 3 12, except that they are each performed under exam room conditions 
immediately before a patient scan is conducted. The conection factors shown in blocks 332 and 
25 334 of Figure 10 are determined from Hxe results of the pre-patient calibration tests. Here, 
correction factors (316, 322) from the factory/PM n\ill target test 304 and the factory/PM open 
air test 3 10 are used along with pre-patient calibration data to determine the pre-patient 
correction factors 118, which are used, in tum, to pre-process raw spectral data from a patient 
scan, as shown, for example, in Figure 1 1. 
30 [0249] Figure 1 1 is a block diagram 340 featuring the spectral data pre-processing component 
1 14 of the tissue characterization system 100 of Figure 1 according to an illustrative embodiment 
of the inventioa In Figure 1 1, 'T" represents the fluorescence data obtained using the UV light 
source 160, '"BBl" represents the broadband reflectance data obtained using the first 188 of the 
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two white light sources 162 and "BB2" represents the broadband reflectance data obtained using 
the second 190 of the two white light sources 162. Blocks 342 and 344 indicate steps undertaken 
in pre-processing raw reflectance data obtained from the tissue usmg each of the two white light 
sources 1 88, 190, respectively. Block 346 indicates steps undertaken in pre-processing raw 
S fluorescence data obtained from the tissue using the UV light source 160. These steps are 
discussed in more detail below. 

[0250] The instrument 102 detailed in Figure 3 features a scanner assembly 180 which 
includes a CCD (charge couple device) detector and spectrograph for collecting fluorescence and 
reflectance spectra from tissue samples. Because a CCD detector is used, the system employs a 
10 calibration procedure to convert a pixel index into wavelength units. Referring to Figure 10, the 
pixel-to-wavelengfh calibration 302 is performed as part of factory and/or preventive 
maintenance calibration procedures 110. 

[0251] In the illustrative embodiment, the tissue classification system 100 uses spectral data 
obtained at wavelengths within a range from about 360 nm to about 720 nm. Thus, the pixel-to- 

15 wavelengdi calibration procedure 302 uses source lig^t Hist produces peaks near and/or within 
the 360 nm to 720 nm range. A mercury lamp produces distinct, usable peaks between about 
365 nm and about 578 nm, and an argon lamp produces distinct, usable peaks between about 697 
rnn and about 740 nm. Thus, the illustrative embodiment uses mercury and argon emission 
spectra to convert a pixel index from a CCD detector into xmits of wavelength (nm). 

20 [0252] First, a low-pressure pen-lamp style mercury lamp is used as source light, and intensity 
is plotted as a function of pixel index. The pixel indices of the five largest peaks are correlated 
to ideal, standard Hg peak positions in units of nanometers. Second, a pen-lamp style argon 
lamp is used as source light and intensity is plotted as a fimction of pixel index. The two largest 
peaks are correlated to ideal, standard Ar peak positions in units of nanometers. 

25 [0253] The seven total peaks provide a set of representative peaks well-distributed within a 
range from about 365 nm to about 738 nm - comparable to tiie range from about 360 nm to 
about 720 nm that is used for data analysis in the tissue classification system 100. The 
calibration procedure in block 302 of Figure 10 includes retrieving the following spectra: a 
spectrum using a mercury lamp as light source, a mercury background ^ectrum (a spectrum 

30 obtained with the mercury source light turned ofE), a spectrum using an argon lamp as light 
source, and an argon background spectrum. The respective Hg and Ar background spectra are 
subtracted from the Hg and Ar spectra, producing the background-corrected Hg and Ar spectra. 
The spectra are essentially noise-firee and require no smoothing. Each of the seven pixel values 
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corresponding to the seven peaks above are determined by finding the centroid of the curve of 
each peak over a +/- 5 pixel range of the maximum as shown in Equation 1 : 

Pmx 

centroid / , , (1) 

Pirn 

where p is pixel value, Ip is ibe intensity at pixel p, and pmax is the pixel value corresponding to 

5 each peak maximum. From the pmax determinations, a polynomial function correlating pixel 
value to wavelength value is determined by perfomiing a least-squares fit of the peak data. In 
one embodiment, the polynomial function is of fourth order. In alternative embodiments, the 
polynomial is of fkst order, second order, third order, fifth order, or higher order. 
[0254] Alternatively to finding pmax by determining the centroid as discussed above, in another 

10 illustrative embodiment the pixel-to-wavelength calibration procedure 302 includes fitting a 
second order polynomial to tiie signal intensity versus pixel index data for each of the seven 
peaks around the maximum +/- 3 pixels (range including 7 pixels); taking tiie derivative of the 
second order polynomial; and finding the y-intercept to determine each pmax- 
[0255] The resulting polynomial function correlating pixel value to wavelength value is 

15 validated, for example, by specifying that the maximum argon peak be located within a given 
pixel range, such as [300:340] and/or that the intensity count at the peak be within a reasonable 
range, such as between 3000 and 32,000 counts. Additionally, the maximum mercury peak is 
validated to be between pixel 150 and 225 and to produce an intensity count between 3000 and 
32,000 coxmts. Next, the maximum difference between any peak wavelength predicted by the 

20 polynomial function and its corresponding ideal (reference) peak is required to be within about 
1.0 imi. Alternatively, oth^ validation criteria may be set. 

[0256] Additional validation procedures may be performed to compare calibration results 
obtained for different units, as well as stability of calibration results over tune. In one illustrative 
embodiment, the pixel4o-wavelength calibration 302 and/or validation is performed as part of 

25 routine preventive maintenance procedures. 

[0257] Since fluorescence and reflectance spectral data that are used as reference data in the 
classification system 100 may be obtained at multiple clinical sites with dififerent individual 
instruments, the illustrative system 100 standardizes spectral data in step 302 of Figure 10 by 
detennining and using values of spectral intensity only at designated values of wavelength. 

30 Spectral intensity values are standardized by interpolating pixel-based intensities such that they 
correspond to wavelengths that are spaced every 1 nm between about 360 nm and about 720 nm. 
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This may be done by linear interpolation of the pixel-based fluorescence and/or reflectance 
values. Other illustrative embodiments use, for example, a cubic spline interpolation procedure 
instead of linear interpolation. 

[0258] In some illustrative embodiments, spectral data acquisition during patient scans and 
5 during the calibration procedures of Figure 10 mcludes the use of a CCD array as part of the 
scanner assembly 180 depicted in Figure 3. The CCD array may contain any number of pixels 
corresponding to data obtained at a given time and at a given interrogation point. In one 
embodiment, the CCD array contains about 532 pixels, including unused leading pixels from 
index 0 to 9, relevant data from index 10 to 400, a power monitor region from index 401 to 521, 

10 and unused trailing pixels from index 522 to 531 . One embodiment includes "power correcting" 
or "power monitor correcting" by scaling raw reflectance and/or fluorescence intensity 
measurements received from a region of a tissue sample with a measure of the intensity of light 
transmitted to the region of the tissue sample. In order to provide the scaling factor, the 
instrument 102 durects a portion of a light beam onto the CCD array, for example, at pixel 

15 indices 401 to 521, and integrates intensity readings over this portion of the array. 

[0259] In one preferred embodunent, both factory/PM 110 and pre-patient 1 16 calibration 
accounts for chromatic, spatial, and temporal variability caused by system interference due to 
external stray light, internal stray light, and electronic background signals. External stray light 
originates from sources external to the instrument 102, for example, examination room lights 

20 and/or a colposcope light. The occurrence and intensity of the effect of external stray light on 
spectral data is variable and depends on patient parameters and the operator's use of the 
mstrument 102. For example, as shown in Figure 8, Ihe farther the probe head 192 rests from the 
speculum 286 in the examination of cervical tissue, the greater the opportunity for room light to 
be present on the cervix. The configuration and location of a disposable component 146 on the 

25 probe head 192 also affects external stray light that reaches a tissue sample. Additionally, if the 
operator forgets to turn off the colposcope light before taking a spectral scan, there is a chance 
that light will be incident on the cervix and affect spectral data obtained. 
[0260] Electronic background signals are signals read from the CCD array when no light 
sources, internal or extemal, are in use. According to the illustrative embodiment, for all 

30 components of the tissue characterization system 1 00 that involve obtaining and/or using spectral 
data, including components 1 10, 1 16, 104, and 1 14 of Figure 1, both extemal stray light and 
electronic background signals are taken into account by means of a background reading. For 
each interrogation point in a spectral scan in which one or more internal light sources are used, a 
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background reading is obtained in which all internal light sources (for example, the Xenon lamps 
and the UV laser) are turned off. According to one feature, the background reading immediately 
precedes the fluorescence and broadband reflectance measurements at each scan location, and 
the system 100 corrects for external stray light and electronic background by subtracting flie 

5 background reading from the conesponding spectral reading at a given interrogation point In 
Figure 10, each calibration test- including 304, 306, 308, 310, 312, 314, 328, and 330 - includes 
obtaining a background reading at each interrogation point and subtracting it from the test 
reading to account for external stray light and electronic background signals. Also, background 
subtraction is a step in the spectral data preprocessing 1 14 methods in Figure 1 1, for the pre- 

10 processing of raw BBl and BB2 reflectance data 342, 344 as well as the pre-processing of raw 
fluorescence data 346. 

[0261] Equation 2 shows the background correction for a generic spectral measurement from a 
tissue sample, StissuefisL«>BSLtm(i»^) • 

SttawKSLCij^) ~ StiuueHSL^L^Cis^) Bk^B^sLCi)^) (2) 

15 where i corresponds to a scan location; X is wavelength or its pixel mdex equivalent; and 

subscripts denote influences on the spectral measurement - where **tissue" represents the tissue 
sample, "ISL" represents internal stray light (internal to the instrument 102), •^ESL" represents 
external stray light, and "EB" represents electronic background. S^^sL-^^-^ih^) is a two- 
dimensional array (which may be power-monitor corrected) of spectral data obtained from the 

20 tissue at each interrogation point (region) i as a function of wavelength X; and BkEB+ESL(i,^) is a 
two-dimensional array representing values of the corresponding background spectral readings at 
each point i as a function of wavelength X. S^jsdiUX) is the background-subtracted spectral 
array that is thereby corrected for effects of electronic background (EB) and external stray light 
(ESL) on the spectral data from the tissue sample. The electronic background reading is 

25 subtracted on a wavelengfh-by-wavelength, location-by-location basis. Subtracting the 

background reading generally does not correct for internal stray light (ISL), as denoted in the 
subscript of Stfa,utwsL(i,^). 

[0262] Internal stray light includes internal cross talk and interaction between the transmitted 
light within the system and the collection optics. For fluorescence measurements, a primary 
30 source of internal stray light is low-level fluorescence of optics internal to the probe 142 and the 
disposable component 146. For reflectance measurements, a primary source of internal stray 
lightis light reflected off of the disposable 146 and surfaces in the probe 142 that is collected 
through the collection optics 200. The positioning of the dii^osable 146 can contribute to the 
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effect of internal stray light on reflectance measurements. For example, the internal stray light 
effect may vary over interrogation points of a tissue sample scan in a non-random, identifiable 
pattem due to the position of the disposable during the test. 

[0263] According to the illustrative embodiment of Figure 10, the factory/PM null target test 

5 304, the factory/PM open air target test 306, the factory/PM custom target test 3 12, the 

factory/PM NIST target test 3 14, the pre^patient null target test 328, and the pre-patient custom 
target test 330 provide correction factors to account for internal stray light effects on 
fluorescence and reflectance spectral measurements. In an alternative illustrative embodiment, a 
subset of these tests is used to account for internal stray light effects. 

10 [0264] The null target test 304, 328, performed in factory/preventive maintenance 110, and 
pre-patient 1 16 calibration procedures, uses a target that has a theoretical diffuse reflectance of 
0%, although the actual value may be higher. Since, at least theoretically, no light is reflected by 
the target, the contribution of internal stray light can be measured for a given internal light 
source by obtaining a spectrum from a region or series of regions of the null target with the 

15 internal light source turned on, obtaining a background spectrum from the null target with the 
internal light source turned off, and background-subtracting to remove any effect of electronic 
background signal or external stray light. The background-subtracted reading is then a measure 
of internal stray li^t. The pre-patient null target test 328 takes into account spatially-dependent 
intemal stray light artifacts induced by the position of a disposable 146, as well as temporal 

20 variability induced, for example, by the aging of the instrument and/or dust accumulation. In 
one embodiment, the factory/PM null target test 304 is used in calculating correction factors 
from other factory and/or preventive maintenance calibration procedures. The null target tests 
304, 328 are not perfect, and improved measurements of the effect of intemal stray light on 
spectral data can be achieved by performing additional tests. 

25 [0265] The open air target test 3 1 0 is part of the fectory preventive maintenance (PM) 

calibration procedure 110 of Figure 10 and provides a complement to the null target tests 304, 
328. The open air target test 310 obtains data in the absence of a target with the intemal light 
sources turned on and all light sources external to the device turned off, for example, in a 
darkroom. The null target test 304, by contrast, does not have to be performed in a darkroom 

30 since it uses a target in place in the calibration port, thereby sealing the instrument such that 
measurements of light from the target are not affected by external light. Although a disposable 
146 is in place during open air test measurements, the factory/PM open air target test 310 does 
not account for any differences due to different disposables used in each patient run. The open 



wo 2004/095359 



PCT/US2004/011820 



-48- 

air measurements are important in some embodiments, however, since they are performed under 
more controlled conditions than pre-patient calibration tests 116, for example, the open air tests 
may be performed in a darkroom. Also, the factory/PM calibration 1 10 measurements account 
for differences between individual instruments 102, as well as the effects of machine aging - 
S both important factors since reference data obtained by any number of individual instruments 
1 02 are standardized for use in a tissue classification algorithm, such as the one depicted in block 
132 of Figure 1. 

[0266] Figures 12, 13, 14, and 15 show graphs demonstrating mean background-subtracted, 
power-monitor-corrected intensity readings from a factory open air target test 3 10 and a null 

10 target test 304 using a BB 1 reflectance white light source and a UV light source (laser). Figure 
12 shows a graph 364 of mean intensity 366 fi-om an open ah: target test over a set of regions as a 
function of wavelength 368 using a BBl reflectance white light source 188 - the 'top" source 
1 88 as depicted m Figures 4, 7, and 8. Figure 13 shows a graph 372 of mean mtensity 366 from 
a null target test over the set of regions as a function of wavelength 368 using the same BBl light 

15 source. Curves 370 and 374 are comparable but there are some differences. 

[0267] Figure 14 shows a graph 376 of mean intensity 378 from an open air target test over a 
set of regions as a function of wavelength 380 using a UV light source, while Figure 15 shows a 
graph 384 of mean intensity 378 from a null target test over the set of regions as a fimction of 
wavelength 380 using the UV light source. Again, curves 382 and 386 are comparable, but there 

20 are some differences between them. Differences between the open air test intensity and null 
target test intensity are generally less than 0.1% for reflectance data and under 1 count/jJ for 
fluorescence data. 

[0268] Accounting for internal stray li^t is more complicated for reflectance measurements 
than for fluorescence measurements due to an increased spatial dependence. The open air target 
25 test measurement, in particular, has a spatial profile that is dependent on the position of the 
disposable. 

[0269] Figure 16 shows a representation 390 of regions of an exemplary scan performed in a 
factory open air target test. The representation 390, shows that broadband intensity readings can 
vary in a non-random, spatially-dependent manner. Other exemplary scans performed in factory 
30 open air target tests show a more randomized, less spatially-dependent variation of intensity 
readings than the scan shown in Figure 16. 

[0270] According to the illustrative embodunent, tiie system 100 of Figure 1 accounts for 
internal stray light by using a combination of tiie results of one or more open au: target tests 310 
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with one or more null target tests 304, 328. In an alternative embodiment, open air target test 
data is not used at all to correct for internal stray light, pre-patient null target test data being used 
instead. 

[0271] Where open air and null target test results are combined, it is helpful to avoid 

5 compounding noise effects from the tests. Figure 17 shows a graph 402 depicting as a function 
of wavelength 406 the ratio 404 of the background-corrected, power-monitor-corrected 
reflectance spectral intensity at a given region using an open air target to the reflectance spectral 
intensity at the region usmg a null target according to an illustrative embodiment of the 
invention. The raw data 407 is shown m Figure 17 fit with a second-order polynomial 412, and 

10 fit with a third-order polynomial without filtering 4 1 0, and with filtering 408. As seen by the 
differences between curve 407 and curves 408, 410, and 412, where a ratio of open air target 
data and null target data are used to correct for internal stray light in reflectance measurements, a 
^ curve fit of the raw data reduces the effect of noise. This is shown in more detail herem with 
resfpect to the calculation of pre-patient corrections 118 m Figure 10. Also evidrat in Figure 17 

IS is tiiat the open air measurement generally differs from the null target measurement, since the 
ratio 404 is not equal to 1, and since the ratio 404 has a distinct wavelength dependence. 
[0272] Figure 18 shows a graph 414 depictmg as a function of wavelength 418 the ratio 416 of 
fluorescence spectral intensity using an open air target to tiie fluorescence spectral intensity 
using a null target according to an illustrative embodiment of the inventioii. The mw data 420 

20 does not display a clear wavelength dependence, except that noise increases at higher 

wavelengths. A mean 422 based on the ratio data 420 over a range of wavelengths is plotted in 
Figure 18. Where a ratio of open air target to null target data is used to correct for internal stray 
light in fluorescence measurements, using a mean value calculated from raw data over a stable 
range of wavelength reduces noise and does not ignore any clear wavelength dependence. 

25 [0273] Figure 1 0 shows correction factors corresponding to open ak 3 1 0 and null target 304, 
328 calibration tests in one embodiment that compensates spectral measurements for intemal 
stray light effects. There are three types of spectral measurements in Figure 10 - fluorescence 
(F) measurements and two reflectance measurements (BBl, BB2) corresponding to data obtained 
using a UV light source and two different white light sources, respectively. The corrections in 

30 blocks 316, 322, and 332 come from the results of the factory/PM null target test 304, the 
fectory/PM open air target test 310, and the pre-patient null target test 328, respectively, and 
these correction factors are applied in spectral data pre-processing (Figure 1 1) to compensate for 
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the effects of internal stray light. These correction factors are described below in terms of this 
embodiment. 

[02741 Block 3 16 in Figure 10 contains correction factors computed from the results of the null 
target test 304, performed during factory and/or preventive maintenance (PM) calibration. The 
5 null target test includes obtaining a one-dimensional array of mean values of spectral data from 
each channel - F, BBl, and BB2 - corresponding to the three different light sources, as shown 
in Equations 3, 4, and 5: 

FCNULLFL = <Wi,U)>» . (3) 

FCNULLBBl = <UB,(i,U)>i (4) 

10 FCNULLBB2 = <Wi,U))i (5) 

where 1^ refers to a background-subtracted, power-monitor-corrected two-dimensional array of 
spectral intensity values; subscript F refers to intensity data obtained usmg the fluorescence UV 
light source; subscripts BBl and BB2 refer to intensity data obtamed using the reflectance BBl 
and BB2 white li^t sources, respectively; i refers to interrogation point "i" on the calibration 

15 target; X refers to a wavelength at which an intensity measurement corresponds or its 

approximate pixel index equivalent; to refers to the fact the measurement is obtained from a 
factory or preventive maintenance test, the *time" the measurement is made; and < X represents a 
one-dunensional array (spectrum) of mean values computed on a pixel-by-pixel basis for each 
interrogation point, i. In this embodiment, a one-dimensional array (spectrum) of fluorescence 

20 values corresponding to wavelengths from A, = 370 nm to A, = 720 nm is obtamed at each of 499 
interrogation points, i. An exemplary scan pattern 202 of 499 intOTogation points appears in 
Figure 5. In the illustrative embodiment, data from an additional interrogation point is obtained 
from a region outside the target 206. Each of the reflectance intensity spectra is obtained over 
the same wavelength range as the fluorescence intensity spectra, but the BBl data is obtained at 

25 each of 250 interrogation points over the bottom half of the target and the BB2 data is obtained 
at each of 249 interrogation points over the top half of the target. This avoids a shadowing effect 
due to the angle at which the light from each source strikes the target during the null target test 
304. Values of the most recent factory or preventive maintenance calibration test, including tiie 
factory/PM null target test 304, are used in spectral data pre-processmg (Figure 1 1) for each 

30 patient scan. 

[0275] The pre-patient null target test, shown in block 328 of Figure 10, is similar to the 
factory/PM null target test 304, except that it is performed just prior to each patient test scan. 
Each pre-patient null target test 328 produces three arrays of spectral data as shown below: 
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(6) 

InU»Bl(iXt') (7) 
I»l.BB2(i,M') (8) 



where t' refers to the fact the measurements are obtained jiist prior to the test patient scan, as 
opposed to during factoiy/PM testing (t,). 

[0276] Block 332 in Figure 10 contains correction fectors from the open air target test 3 10, 
preformed during fectory and/or preventive maintenance (PM) calibration 110. The open air 
target test is performed with the disposable in place, in the absence of a target, with the internal 
light sources turned on, and with all li^t sources external to the device turned off. The open air 
target test 3 10 includes obtaining an anay of spectral data values from each of the three channels 
- F, BBl, and BB2 - as shown below: 

MiM) (9) 
I«*Bi(i^l.) (10) 
l^jaCuKQ (11) 
[0277] In each of items 9, 1 0, and 1 1 above, L refers to a background-subtracted, power- 
monitor-corrected array of spectral intaisity values; i runs from interrogation points 1 to 499; 
and X runs from 370 nm to 720 nm (or fbe approximate pixel index equivalent). 
[0278] Accordmg to the illustrative embodiment, correction for internal stray light makes use 
of both null target test results and open air target test results. Correction factors in block 322 of 
Figure 10 use results from the factory/PM null target test 304 and factory/PM open air target test 



310. The correction fectors in block 322 are computed as follows: 

sFCOFL = [ (U}Xt.)\ I <Uag). ].»,x.,«».«o«. (12) 

FCOBBl = fitted form of <UBB,(i,U)>. / OUbiCU)), (13) 

FCOBB2 = fitted fomi of <IoM,B2(iAg>i / <UB:(i,U)>i (14) 



where < >i represents a spectrum (1 -dimensional array) of mean values computed on a pixel-by- 
pixel basis for each interrogation point i, and where < >i / <)i represents a spectrum (1- 
dunensional array) of quotients (ratios of means) computed on a pixel-by-pixel basis for each 
interrogation point i. The correction fector sFCOFL in Equation 12 is a scalar quantity 
representing the mean value of Ac 1-dimensional array in brackets [ ] across pixel indices 
correspondmg to the wavelength range of about 375 nm to about 470 nm. 
[0279] Figure 18 shows an example value of sFCOFL 422 evaluated using a set of mean open 
air spectral data and mean null target spectral data. Large oscillations are damped by using the 



wo 2004/095359 



PCT/US2004/011820 



-52- 

mean in Equation 12. Other wavelength ranges can be chosen instead of the wavelength range of 
about 375 nm to about 470 nm. 

[0280] The one-dimensional arrays, FCOBBl and FCOBB2, are obtained by curve-fitting the 
spectra of quotients in Equations 13 and 14 with second-order polynomials and determining 
values of the curve fit corresponding to each pixel. Figure 17 shows an example curve fit for 
FCOBBl (412). Unlike the fluorescence measurements, fliere is wavelength dependence of this 
ratio, and a curve fit is used to properly reflect this wavelength dependence without introducing 
excessive noise in following computations. 

[0281] Block 332 m Figure 10 contains correction factors using results from the pre-patient 
null target test 328, as well as the most recent factory/PM null target test 304 and open air target 
test 310. The correction factors in block 332 are computed as follows: 

SLFL = sFCOFL • (I^(i,X,f)>i (15) 
SLBBl = FCOBBl • CUBB.(i,MO>i (16) 
SLBB2 = FCOBBl • <}^{iXtl)i (17) 
where Equation 15 represents multiplying each value in the fluorescence mean pre-patient null 
target spectrum by the scalar quantity sFCOFL from Equation 12; Equation 16 represents 
multiplying corresponding elements of the mean pre-patient null target BBl spectrum and the 
one-dimensional array FCOBBl from Equation 13; and Equation 17 represents multiplying 
corresponding elements of the mean pre-patient null target BB2 spectrum and the one- 
dimensional array FCOBB2 from Equation 14. Each of SLFL, SLBBl, and SLBB2 is a one- 
dimensional array. 

[0282] The correction factors m block 332 of Figure 10 represent the contribution due to 
internal stray light (ISL) for a given set of spectral data obtained from a given patient scan. 



Combining equations above: 

SLFL= [ <W(iM))i / <Int^(i,U)>i ] mean, X - 375 cm to 470 nro • MXt'))i (18) 

SLBBl - [ <Io.BBi(iM)>i / <Wi(iM))i W ' (Ubi W'))i (19) 

SLBB2 = [ aa3B2(i,U)>i / a«t3B2(iM)). • < W(i,^t'))i (20) 



[0283] Alternative internal stray light correction factors are possible. For example, in one 
alternative embodiment, the scalar quantity in Equation 1 8 is replaced with the value 1 .0. In one 
alternative embodiment, the first term on the right side of eiflier or both of Equation 19 and 
Equation 20 is replaced with a scalar quantity, for example, a mean value or the value 1 .0. 
[0284] Spectral data preprocessmg 1 14 as detailed in Figm-e 1 1 includes compensating for 
internal stray light effects as measured by SLFL, SLBBl and SLBB2. In one embodiment, a 
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patient scan includes the acquisition at each interrogation point in a scan pattern (for example, 
the 499-point scan pattern 202 shown in Figure 5) of a set of raw fluorescence intensity data 
using the UV light source 160, a first set of raw broadband reflectance intensity data using a first 
white light source (162, 188), a second set of raw broadband reflectance intensity data using a 

5 second vMe light source (162, 192), and a set of raw background intensity data using no 
internal light source, vrfiere each set of raw data spans a CCD pixel index corresponding to a 
wavelength range between about 370 nm and 720 nm. In another anbodiment, Ae wavelengfli 
range is from about 370 nm to about 700 nm. In another embodiment, the wavelength range is 
&om about 300 nm to about 900 nm. Other embodiments include the use of different 

10 wavelength ranges. 

[0285] The raw background mtensity data set is represented as the two-dimensional array 
Bl%nd[] in Figure 1 1 . Spectral data processing 1 14 includes subtracting the background array, 
BkgndO, fiom each of the raw BBl, BB2, and F arrays on a pixel-by-jrixel and location-by- 
location basis. This accounts at least for electronic background and external stray li^t effects, 

15 and is shown as item #1 in each of blocks 342, 344, and 346 m Figure 11. 

IP2861 Also, each CCD array containing ^ectral data includes a portion for monitoring the 
power output by the light source used to obtain the spectral data, hi one embodiment, the 
intensity values in this portion of each array are added or integrated to provide a one-dimensional 
anray of scalar values, sPowerMonitorQ, shown in Figure 1 1 . Spectral data pre-processing 1 14 

20 further includes dividing each element of the background-subtracted arrays at a given 
interrogation point by the power monitor scalar correction fector in sPowerMonitorQ 
corresponding to the given interrogation point. This allows the expression of spectral data at a 
given wavelengtti as a ratio of received ligjit intensity to transmitted light intaisity. 
[0287] Spectral data pre-processmg 1 14 fiffther includes subtracting each of the stray light 

25 background arrays - SLBBl, SLBB2, and SLFL - fix)m its corresponding background- 
corrected, power-monitor-corrected spectral data array - BBl, BB2, and F - on a pixel-by-pixel, 
location-by-location basis. This accounts for chromatic, temporal, and spatial variability effects 
of intemal stray light on the spectral data. 

[0288] The remaining steps in blocks 342 and 344 of the spectral data pre-processing block 
30 diagram 340 of Figure 1 1 include further factory, preventive mamtenance (?M) and/or pre- 

patient calibration of reflectance (BBl, BB2) measurements usmg one or more targets of known, 
non-zero diffuse reflectance, hi the embodiment shown in Figure 10, this calibration uses results 
fit)m the fectory/PM custom target test 312, the factory/PM NIST-standard target test 3 14, and 
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the pre-patient custom target test 330. These calibration tests provide correction factors as 
shown in blocks 324, 326, and 334 of Figure 10, that account for chromatic, temporal, and 
spatial sources of variation in broadband reflectance spectral measurements. These sources of 
variation include temporal fluctuations in the illumination source, spatial inhomogeneities in the 
S illummation source, and chromatic aberration due to the scanning optics. The broadband 

reflectance calibration tests (312, 314, 330) also account for system artifacts attributable to both 
transmitted and received light, since these artifacts exist in both test reflectance measurements 
and known reference measurements. 

[0289] According to the illustrative embodiment, reflectance, R, computed from a set of 

10 regions of a test sample (a test scan) is expressed as in Equation 21 : 

R = [Measurement / Reference Target] • Reflectivity of Reference Target (21) 
where R, Measurement, and Reference Target refer to two-dimensional (wavelength, position) 
arrays of background-corrected, power-corrected and/or intemal-stray-light-corrected reflectance 
data; Measurement contains data obtained firom the test sample; Reference Target contains 

15 data obtained from the reference target; Reflectivity of Reference Target is a known scalar value; 
and division of the arrays is performed in a pixel-by-pixel, location-by-location maimer. 
[0290] The factory/PM NIST target test 314 uses a 60%, NIST-traceable, spectrally flat diffuse 
reflectance target in the focal plane, aligned in the instrument 102 represented in Figure 3. The 
NIST target test 314 includes performing four scans, each of which proceed with the target at 

20 different rotational orientations, perpendicular to the optical axis of the system. For example, the 
target is rotated 90'' from one scan to the next. The results of the four scans aie averaged on a 
location-by-location, pixel-by-pixel basis to r^ove spatially-dependent target artifects 
(speckling) and to reduce system noise. The goal is to create a spectrally clean (low noise) and 
spatially-flat data set for application to patient scan data» In one embodunent, the NIST target 

25 test 3 14 is performed only once, prior to instrument 102 use in the field (factory test), and thxis, 
ideally, is temporally invariant. 

[0291] The custom target tests 3 12, 330 use a custom-made target for both factory and/or 
preventive maintenance calibration, as well as pre-patient calibration of reflectance data. The 
custom target is a 10% diffuse reflective target with phosphorescent and/or fluorescent portions 
30 used, for example, to align the ultraviolet (UV) light source and/or to monitor the stability of 
fluorescence readings between preventive maintenance procedures. Figure 19 is a photograph of 
the custom target 426 according to an illustrative embodiment. In Figure 19, the target 426 
includes a portion 428 fliat is about 10% difiuse reflective material, with four phosphorescent 
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plugs 430, 432, 434, 436 equally-spaced at the periphery and a single fluorescent plug 438 at the 
center. As a result of the plugs, not all scan locations in the scan pattern 202 of Figure 5, as 
applied to the custom target test 426, accurately measure the 10% reflective portion. Thus, a 
mask provides a means of filtering out the plug-influenced portions of the custom target 426 

S during a custom target calibration scan 312, 330. 

[0292] Figure 20 is a representation of such a mask 444 for the custom target reflectance 
calibration tests 312, 330. Area 445 in Figure 20 corresponds to regions of the custom target 426 
of Figure 19 that are not affected by the plugs 430, 432, 434, 436, and which, therefore, are 
usable in the custom target reflectance calibration tests 312, 330. Areas 446, 448, 450, 452, and 

10 454 of Figure 20 correspond to regions of the custom target 426 that are affected by the plugs, 
and which are masked out in the custom target calibration scan results. 
[0293] In the illustrative embodiment, the factory/PM NIST target test 3 14 provides 
reflectance calibration data for a measured signal from a test sample (patient scan), and the test 
sample signal is processed according to Equation 22: 

15 R(iXtl = MUW I Ui,U)] • 0.6 (22) 

[0294] Where R, I„, and Ife are two-dimensional arrays of background-corrected, power- 
corrected reflectance data; R contains reflectance intensity data from the test sample adjusted 
according to the reflectance calibration data; I„ contains reflectance intensity data from the 
sample, Ifc contains reflectance intensity data from the factory/PM NIST-standard target test 314, 

20 and 0.6 is the known reflectivity of the NIST-standard target Equation 22 presumes the spectral 
response of the illumination source is temporally invariant such that the factory calibration data 
from a given unit does not change with time, as shown in Equation 23 below: 

I<b(t') = Iic(to) (23) 
However, the spectral lamp ftinction of a xenon flash lamp, as used in the illustrative 

25 embodiment as the white light source 1 62 in the instrument 1 02 of Figure 3, is not invariant over 
time. 

[0295] The illustrative reflectance data spectral preprocessing 1 14 accounts for temporal 
variance by obtaining pre-patient custom target test (330) reflectance calibration data and using 
the data to adjust data from a test sample, I„„ to produce adjusted reflectance R, as follows: 
30 R(i,M') - [I»(i,X,f ) / <Icp(i,X,t')>, 3 ' 0.1 (24) 

where masked, mean reflectance intensity data from the pre-patient custom target test 330 with 
10% diffuse reflectivity, <Icp(i,MO>i . replaces Ifc(i,A,,f ) in Equation 22. Smce the pre-patient 
custom target test data is updated before every patient exam, the temporal variance effect is 
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dimimshed or eliminated. In other illustrative embodiments, various other reference targets may 
be used in place of the custom target 426 shown in Figure 19. 

[0296] The system 100 also accounts for spatial variability in the target reference tests of 
Figure 10 in pre-processing reflectance spectral data. Illustratively, spatial variability in 
5 reflectance calibration target intensity is dependent on wavelength, suggesting chromatic 
aberrations due to wavelength-dependence of transmission and/or collection optic efficiency. 
[0297] The illustrative reflectance data spectral preprocessing 1 14 accounts for tiiese 
chromatic and spatial variability effects by obtaining reflectance calibration data and using the 
data to adjust data from a test sample, I„„ to produce adjusted reflectance R, as follows: 

10 R(at') = MhKV) I (UUt')). ] • [<Ifc(i,U)>. / UiXto) ] • 0.1 (25) 

Equation 25 accounts for variations of the intensity response of the lamp by applying the pre- 
patient custom-target measurements - which are less dependent on differences caused by the 
disposable - in correcting patieat test sample measurements. Equation 25 also accounts for the 
spatial response of the illumination source by applying the fectory NIST-taiget measurements in 

IS correcting patient test sample measurements. 

[0298] In an alternative illustrative embodiment, the NIST-target test 3 14 is performed as part 
of pre-patient calibration 1 16 to produce calibration data, Ifc(i,>-,t'), and Equation 22 is used in 
processing test reflectance data, where the quantity Ifc(i,X,t') replaces the quantity Ifc(i,X,to) in 
Equation 22. According to this illustrative embodiment, the test data pre-processing procedure 

20 1 14 mcludes both factory/PM calibration 110 results and pre-patient calibration 1 16 results in 
order to maintain a more consistent basis for the accumulation and use of reference data from 
various individual units obtained at various times frnxa. various patients in a tissue 
characterization Systran. Thus, this illustrative embodiment uses Equation 26 below to adjust 
data from a test sample, I„, to produce adjusted reflectance R, as follows: 

25 R(i,M') = [In,(i.M') / <Ifc (iAt'))i ] • IM^XQ), I Ui,U) ] • 0.6 (26) 

where the NIST-standard target test 3 14 is performed both as a factory/PM test 1 10 (t.) and as a 
pre-patient test 1 16 (f). 

[0299] According to the illustrative embodiment, it is preferable to combine calibration 
standards with more than one target, each having a different diflRise reflectance, since calibration 
30 is not tiien tied to a smgle reference value. Here, processmg using Equation 25 is preferable to 
Equation 26. Also, processing via Equation 25 may allow for an easier pre-patient procedure, 
since the custom target combines functions for both fluorescence and reflectance system set-up, 
avoiding t^e need for an additional target test procedure. 
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[0300] Values of tiie custom target reflectance in a given individual instrument 102 vary over 
time and as a function of wavelength. For example. Figure 21 shows a graph 458 depicting as a 
function of wavelength 462 a measure of the mean reflectivity 460, Rep, of the 10% diffuse target 
426 of Figure 19 over the non-masked regions 445 shown in Figure 20, obtained using the same 

5 instrument on two different days. R, is calculated as shown in Equation 27: 

R.p(^)= [ap(i^O>./<Ift(i^t.)>,] -R^, (27) 
where Rfi. = 0.6, the diffuse reflectance of the NIST-traceable standard target Values of R^ vary 
as a function of wavelength 462, as seen in each of curves 464 and 466 of Figure 21 . Also, there 
is a shift from curve 464 to curve 466, each obtained on a different day. Similarly, values of Rep 

10 vary among different instrument units. Curves 464 and 466 show that R^, varies with wavelength 
and varies from 0.1; thus, assuming R«p = 0.1 as in Equation 25 may introduce inaccuracy. 
[0301] Equation 25 can be modified to account for this temporal and wavelength dependence, 
as shown in Equation 28: 

R(i^t') = MiXn I OpCi^t')), ] ■ [(UiM)>i / Ui.U) 1 • R.^ (28) 

15 yAistt "RivfiiM is an array of values of a second-order polynomial curve fit of R^ shown in 

Equation 27. The polynomial curve fit reduces Ae noise in the R.P array. Other curve fits may 
be used alternatively. For example. Figure 22A shows a graph 490 depicting, for seven 
individual instruments, curves 496, 498, 500, 502, 504, 506, 508 of sample reflectance intensity 
using the BBl white light source 188 as depicted in Figures 4, 7 and 8 graphed as fimctions of 

20 wavelength 494. Each of the seven curves represents a mean of reflectance intensity at each 
wavelength, calculated using Equation 25 for regions confirmed as meteq^lasia by impression. 
Figure 22B shows a graph 509 depicting corresponding curves 510, 512, 514, 516, 518, 520, 522 
of test sample reflectance intensity calculated usmg Equation 28, where R, varies with time and 
wavelength. The variability between individual instrument units decreases when using measured 

25 values for Rep as in Equation 28 rather than as a constant value. The variability between 
reflectance spectra obtained from samples having a common tissue-class/state-of-health 
classification, but using different instrument units decreases when using measured values for R«p 
as in Equation 28 rather than a constant value as in Equation 25. 
[0302] In an alternative embodiment, processing of reflectance data includes applying 

30 Equation 28 without fibrst fitting R,p values to a quadratic polynomial. Thus, processing is 
performed in accordance with Equation 29 to adjust data from a test sanq)le, I„ to produce 
adjusted reflectance R, as follows: 

R(i,X,t') = PLa,^') / <Icp(i.X,t'))i ] • [<Ui,U». / IftCiM) 1 • Rep (29) 
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[0303] Applying Bquation 29, however, introduces an inconsistency in the reflectance spectra 
at about 490 nm, caused, for example, by the intensity from the 60% reflectivity factory 
calibration target exceeding the Unear range of the CCD array. This can be avoided by using a 
darker factory calibration target in the factory NIST target test 3 14, for example, a target having 
5 a known diffuse reflectance from about 10% to about 30%. 

[0304] Results &om the fectory/PM custom target test 3 12, the fectory/PM NIST target test 
314, and the pre-patient custom target test 330 provide the correction factors shown in blocks 
324, 326, and 334, respectively used in preprocessing reflectance data from a patient scan using 
the BBl white light source 188 and the BB2 white light source 190 shown in Figures 4, 7, and 8. 

10 Correction factors in block 324 represent background-subtracted, power-monitor-corrected 
(power-corrected), and null-target-subtracted reflectance data from a given factory/PM custom 
target test 3 12 (qp) and are shown in Equations 30 and 3 1 : 

FCCTMMBBl = <I^,(iM)X«-w -FCNULLBBl (30) 
FCCTMMBB2 = <I.p^(iM)X««toi - FCNtJLLBBl (31) 

15 where FCNULLBBl and FCNULLBB2 are given by Equations 4 and 5, and < X ^ represents 
a one-dimensional array of mean data computed on a pixel-by-pixel basis in regibns of area 445 
of the scan pattern 444 of Figure 20. 

[0305] Correction factors in block 326 of Figure 10 represent ratios of background-subtracted, 
power-corrected, and nxill-target-subtracted reflectance data from a factory/PM custom target test 
20 312 (cp) and a fectory/PM NIST standard target test 3 14 (fc) and are shown in Equations 32, 33, 
and 34: 

FCBREFin= (l^,(iXt.)«.^» - FCNULLBBl), (32) 

IfeBBi(iM).v8.f4 - FCNULLBBl 
FCBREF2a= <lMB2(iM).vgof4-FCNULLBB2X (33) 

25 lMB2(i,U)«vB,f4 - FCNULLBB2 

CALREF = [0.5-(FCCTMBBl/<FCBREFin>0 + (34) 
(FCCTMBB2/(FCBREF2n)0 li^n^ 

where values of the two-dimensional arrays la^, and Ift^Bi are averages of data using the target at 
each of four positions, rotated 90° between each position; and all divisions, subtractions, and 
30 multiplications are on a location-by-location, pixel-by-pixel basis. The correction factor, 

CALREF, is a one-dimensional array of values of the quantity in brackets [ ] on the right side of 
Equation 34, interpolated such that they correspond to wavelengths at 1-mn intervals between X = 
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360 ran and X = 720 ran. The interpolated values are then fit with a quadratic or other polynomial 
to reduce noise. 

[0306] Correction factors in block 334 of Figure 10 represent background-subtracted, power- 
corrected, intemal-stray-light-corrected reflectance data from a pre-patient custom target test 330 
S (cp) and are given in Eqiiations 35 and 36 as follows: 



where SLBBl and SLBB2 are as shown in Equations 1 9 and 20. 

[0307] Steps #4, 5, and 6 in each of blocks 342 and 344 of the spectral data pre-processing 
10 block diagram 340 of Figure 1 1 include processing patient reflectance data using the correction 
factors from blocks 324, 326, and 334 of Figure 10 computed using results of the factory/PM 
custom target test 3 12, the factory/PM NIST standard target test 3 14, and the pre-patient custom 
target test 330. 

[0308] In step #4 of block 342 in Figure 1 1, the array of background-subtracted, power- 

15 corrected, intemal-stmy-ligjit-subtracted patient reflectance data obtained using Ike BBl light 
source is multiplied by the two-dimensional array correction factor, FCBREF1[], and then in 
step #5, is divided by the correction factor BREFMBBl. After filtering usmg, for example, a 5- 
point median filter and a second-order 27-point Savitsky-Golay filter, the resulting array is 
linearly interpolated using results of the wavelength calibration step 302 in Figure 10 to produce 

20 a two-dimensional array of spectral data corresponding to wavelengths ranging firom 360 nm to 
720 nm in 1-nm increments at each of 499 interrogation points of the scan pattern 202 shown in 
Figure 5. This array is multiplied by CALREF in step #6 of block 342 in Figure 1 1 , and pre- 
processing of the BBl spectral data in this embodunent is complete. 
[0309] Steps #4, 5, and 6 in block 344 of Figure 1 1 concern processing of BB2 data and is 

25 directly analogous to the processing of BB 1 data discussed above. 

[0310] Steps #4 and 5 in block 346 of Figure 1 include processing fluorescence data using 
factory/PM-level correction factors, applied after background correction (step #1), power 
monitor correction (step #2), and stray Ught correction (step #3) of fluorescence data firom a test 
sample. Steps #4 and 5 include application of correction factors sFCDYE and IRESPONSE, 

30 which come firom the factory/PM fluorescent dye cuvette test 306 and the factory/PM tungsten 
source test 308 in Figure 10. 

[0311] The fiactory/PM tungsten source test 308 accounts for the wavelength response of the 
collection optics for a given instrument unit The test uses a quartz tungsten halogen lamp as a 



BREFMBBl = <I^,(i,X,f ) - SLBB1>| 
BREFMBB2 = (Icp3«(i,M') - SLBB2>, 



(35) 
(36) 
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light source. Emission from the tungsten jBlament approximates a blackbody emitter Planck's 
radiation law describes the radiation emitted into a hemisphere by a blackbody (BB) emitter: 

Wbb(^) = [a • (CE)] / [ • {exp(bA,T) - 1}] (37) 
where a = 27chc^ = 3.742 x 10^^ [W(mn//cm^]; b = hc/k = 1,439 x 10^ [(nm)K]; T is source 
5 temperature; CE is a fitted parameter to account for collection efficiency; and both T and CE are 
treated as variables determined for a given tungsten lamp by curve-fitting emission data to 
Equation 37. 

[0312] The lamp temperature, T, is determined by fitting NIST-traceable source data to 
Equation 37. Figure 23 shows a graph 582 depicting the spectral hxadiance 584, W^nsD^y of a 

10 NIST-traceable quartz-tungsten-halogen lamp, along with a curve fit 590 of the data to the model 
in Equation 37 for blackbody irradiance, Wbb. Since the lamp is a gray-body and not a perfect 
blackbody, Equation 37 includes a proportionality constant, CE. This proportionality constant 
also accounts for the "collection efficiency" of the setup in an instrument 102 as depicted in the 
tissue characterization system 1 00 of Figure L In the illustrative embodiment, the target fi'om 

15 which measurements are obtained is about 50-cm away &om the lamp and has a finite collection 
cone that subtends a portion of the emission hemisphere of the lamp. Thus, while Wbb(?^) in 
Equation 37 has imits of [W/nm], calibration values for a given lamp used in the instrument 102 
in Figure 1 has units of [W/cm^-nm at 50 cm distance]. The two calibration constants, CE and T, 
are obtained for a given lamp by measuring the intensity of the given lamp relative to the 

20 intensity of a NIST-calibrated lamp using Equation 38: 

Wiamp = Ptomp / IraSTIainp ] ' WNisThmp (38) 

Then, values of T and CE are determined by plotting W^^ versus wavelength and curve-fitting 
using Equation 37. The curve fit provides a calibrated lamp response, lumpQ^), to which the 
tungsten lamp response measured during factory/PM testing 308 at a given interrogation point 

25 and usmg a given instrument, Siamp(i,A,), is compared. This provides a measure of "instrument 
response", IR(i,A,), for the given point and the given instrument, as shown in Equation 39: 

IR(iA) = Su.p(i,X)/I,^(^) (39) 
[0313] The factory/PM tungsten source test 308 in Figure 10 includes collecting an intensity 
signal firom the tungsten lamp as its light reflects off an approximately 99% reflective target 

30 The test avoids shadowing effects by altemately positioning the tungsten source at each of two 
locations - for example, on either side of the probe head 192 at locations corresponding to the 
white light source locations 188, 190 shown in Figure 8 - and using the data for each given 
interrogation point corresponding to the source position where the given point is not in shadow. 
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[0314] Once the instrument response measure, IR(i,X), is obtained, a correction factor is 
determined such that its value is normalized to unity at a given wavelength, for example, at 3l = 
500 nm. Thus, the distance between the lamp and the detecting aperture, the photoelectron 
quantum efficiency of the detector, and the reflectivity of the target do not need to be measured. 

5 [0315] Accordmg to the illustrative embodiment, the fluorescence component of the spectral 
data pre-processing 1 14 of the system ICQ of Figure 1 corrects a test fluorescence mtensity 
signal, SfGjX), for individual instrument response by applying Equation 40 to produce IfCI^), the 
instrument-response-corrected fluorescence signal: 

UUX) = S,(i,X) - [{500 IR(i,^)}/{^- IR(i,500)} ] (40) 

10 where IR(i,500) is the value of the instrument response measure IR at point i and at wavelength X 
= 500 nm; and where the term X/SOO converts the fluorescence intensity from energetic to 
photometric units, proportional to fluorophore concentration. In one embodiment, the 
differences between values of IR at different interrogation pomts is smaU, and a mean of IR(X) 
over all interrogation points is used in place of IR(i,X) in Equation 40. 

15 [0316] The fluorescent dye cuvette test 306 accounts for variations in the efficiency of the 
collection optics 200 of a given instrument 102. Fluorescence collection efficiency depends on a 
number of factors, including the spectral response of the optics and detector used. In one 
embodiment, for example, the collection efficiency tends to decrease when a scan approaches the 
edge of the optics. A fluorescent dye cuvette test 306, performed as part of factory and^or 

20 preventive maintenance (PM) calibration, provides a means of accountmg for efficiency 
differences, 

[0317] An about 50-mm-diameter cuvette filled with a dye solution serves as a target for the 
fluorescent dye cuvette test 306 to account for collection optic efficiency variation with 
interrogation point position and variation between different units. The fectory/PM dye-filled 

25 cuvette test 306 mcludes obtammg the peak intensity of the fluorescence intensity signal at each 
interrogation point of the dye-filled cuvette, placed in the calibration target port of the instrument 
102, and comparing it to a mean peak mtensity of the dye calculated for a plurality of units. 
[0318] Illustratively, a calibrated dye cuvette can be prepared as follows. First, the 
fluorescence emission of a 10-mm-pathlength quartz cuvette filled with ethylene glycol is 

30 obtained. The ethylene glycol is of 99+% spectrophotometric quality, such as that provided by 
Aldrich Chemical Company. The fluorescence emission reading is verified to be less than about 
3000 counts, particularly at wavelengths near the dye peak mtensity. An approximately 2.5 x 
1 0"^ moles/L solution of coumarm-5 1 5 in ethylene glycol is prepared. Coumarin-5 1 5 is a 
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powdered dye of molecular weight 347, produced, for example, by Exciton Chemical Company. 
The solution is diluted with ethylene glycol to a final concentration of about 1.2 x 10'^ moIes/L. 
Then, a second lO-mm-pathlength quartz cuvette is filled with the coumarin-515 solution, and an 
emission spectrum is obtained. The fluorescence emission reading is verified to have a 
S maximum between about 21 0,000 coxmts and about 250,000 counts. The solution is titrated with 
either ethylene glycol or concentrated coumiarm-S15 solution until the peak lies in this range. 
Once achieved, 50-mm-diameter quartz cuvettes are filled with the titrated standard solution and 
flame-sealed. 

[0319] A correction factor for fluorescence collection efficiency can be determined as follows. 
10 First, the value of fluorescence intensity of an instrument-response-corrected signal, lF(i,X,), is 

normalized by a measure of the UV light energy delivered to the tissue as in Equation 41 : 

F^Ci,^) = [ Ip(i,X) / P„(i) ] • [ P„ / E^ ]^ (41) 

where FtO,^) is the instniment-response-corrected, power-monitor-corrected fluorescence 

intensity signal; Pi„(i) is a power-monitor reading that serves as an indirect measure of laser 
15 energy, determined by integrating or adding intensity readmgs fi-om pixels on a CCD array 

corresponding to a portion on which a beam of the output laser light is directed; and [ P„ / E^j 

]fam is tiie ratio of power monitor reading to output laser energy determined during factory 

calibration and/or preventive maintenance (FC/PM). 

[0320] Next, the illustrative embodiment includes obtaining the fluorescence intensity 
20 response of a specific unit at a specific interrogation point (region) in its scan pattem using a 
cuvette of the titrated coumarin-515 dye solution as the target, and comparing that response to a 
mean fluorescence intensity response calculated for a set of units, after accounting for laser 
energy variations as in Equation 41 . Equation 42 shows a fluorescence collection efficiency 
correction factor for a given unit applied to an instrument-response-corrected fluorescence 
25 signal, lf{i,X), along with the energy correction of Equation 41 : 



^ I Instruments 



(42) 



PM 



where loJ^iX^ is the peak measured fluorescence intensity at interrogation position i using the 
dye-filled cuvette, as shown in Figure 31; ^ is the wavelength (or its approximate pixel index 
equivalent) corresponding to the peak intensity; and the quantity in brackets < )bu«mm is the mean 
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power-corrected intensity at interrogation point 251, corresponding to the center of the 
exemplary scan pattern of Figure 5, calculated for a plurality of units. 
[0321] The fluorescence collection efficiency tends to decrease when the scans approach the 
edge of the optics. Figure 24 shows typical fluorescence spectra from the dye test 306. The 
graph 614 in Figure 24 depicts as a function of wavelength 618 the fluorescence intensity 616 of 
the dye solution at each region of a 499-point scan patteriL The curves 620 aU have 
approximately the same peak wavelength, Xp, but the mayirn^iin fluorescence intensity values 
vary. 

[0322] Figure 25 shows how the peak fluorescence intensity (intensity measured at pixel 131 
corresponding approximately to Xp) 624, determined in Figure 24, varies as a function of scan 
position (interrogation point) 626. Oscillations are due at least in part to optic scanning in the 
horizontal plane, while the lower frequency frown pattern is due to scan stepping in the vertical 
plane. According to the illustrative embodiment, curves of the fluorescence intensity of the dye 
cuvette at approximate peak wavelength are averaged to improve on the signal-to-noise ratio. 
[0323] Equation 42 simplifies to Equations 43 and 44 as follows: 



= M^.FCDYE(i) 



Instruments 



(43) 



PM 



(44) 



The tenn, [ P„ / E^, ]pm, drops out of equation 42. Variations in laser energy measurements 
become less important as the energy is averaged over multiple measurements made on many 
instruments. 

[0324] In Figure 10, the correction factor sFCDYE in block 3 18 is a one-dimensional scalar 
array and is calculated using Equation 45: 



r 



sFCDYE = 



P„i251) 'e. 



/ Instruments 



(45) 



Here, values of Ii>^i,X,) are background-subtracted, power-coirected, and null-taiget-subtracted. 
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[0325] In Figure 10, the correction factor BRESPONSE in block 320 is a one-dimensional 
array and is calculated using the results of the factory/PM tungsten source test 308, as in 
Equation 46: 

IRESPONSE = [{500-IR(i,X)}/{X- IR(i,500)} ] (46) 
5 where IR(i,500) is the value of the instrument response measure IR given in Equation 39 at point 
i and at wavelength X = 500 nm. 

[0326] Steps #4 and 5 in block 346 of the fluorescence spectral data pre-processing block 
diagram 340 of Figure 1 1 include processmg fluorescence data usmg sFCDYE and 
IRESPONSE as defined in Equations 45 and 46. The fluorescence data pre-processing proceeds 

10 by background-subtracting, power-correcting, and stray-light-subtracting fluorescence data fi'om 
a test sample using Bkgnd[], sPowerMonitor[l, and SLFL as shown in Steps #1, 2, and 3 in 
block 346 of Figure 1 1 . Then, the result is multiplied by sFCDYE and divided by IRESPONSE 
on a pixel-by-pixel, location-by-location basis. Next, the resulting two-dimensional array is 
smoothed iismg a 5-point median filter, then a second-order, 27-point Savitsky-Golay filter, and 

15 interpolated using the pixel-to-wavelength conversion determined in block 302 of Figure 1 0 to 
produce an array of data corresponding to a spectrum covering a range firom 360 nm to 720 ma 
at 1-nm intervals, for each of 499 interrogation points of the scan pattern. 
[0327] As a further feature, the stability of fluorescence intensity readings are monitored 
between preventive maintenance procedures. This may be performed prior to each patient scan 

20 by measuring the fluorescence intensity of the center plug 438 of the custom target 426 shown in. 
Figure 19 and comparing the result to the expected value firom the most recent preventive 
maintenance test If the variance firom the expected value is significant, and/or if the time 
betwera successive preventive maintenance testing is greater than about a month, the following 
correction &ctor may be added to those in block 346 of Figure 1 1 : 



"4(251,1^)" 




L _ 


PM 


7,(25 




L ^«(251) _ 


PP 



where PM denotes preventive maintenance test results; PP denotes pre-patient test results; 
Ict(25 1 , Xp) is flie fluorescence peak intensity reading at scan position 25 1 (center of the custom 
target) at peak wavelength \; and Pm is the power monitor reading at scan position 251 . 
[0328] The spectral data pre-processmg 1 14 in Figure 1 1 fiirther includes a procedure for 
30 characterizing noise and/or applying a threshold specification for acceptable noise performance. 
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Noise may be a significant factor in fluorescence spectral data measurements, particularly where 
the peak fluorescence intensity is below about 20 counts/tJ (here, and elsewhere in this 
specification, values expressed in terms of counts/^J are interpretable in relation to the mean 
fluorescence of normal squamous tissue being 70 ct/pJ at about 450 nm). 

5 [0329] The procedure for characterizing noise includes calculating a power spectrum for a null 
target background measurement The null target background measurement uses a mdl target 
having about 0% reflectivity, and the measurement is obtained with internal lights off and 
optionally with all extemal lights turned off so that room lights and other sources of stray light 
do not affect the measurement Preferably, the procedure includes calculating a mean null target 

10 background spectrum of the individual null target background spectra at all interrogation points 
on the target - for example, at all 499 points of the scan pattern 202 of Figure 5. Then, the 
procedure subtracts the mean spectrum from each of the individual null target background 
spectra and calculates the Fast Fourier Transform (FFT) of each mean-subtracted spectrum. 
Then, a power spectrum is calculated for each FFT spectrum and a mean power spectrum is 

15 obtained. 

[0330] Figure 26 shows a graph 678 depicting exemplary mean power spectra for various 
individual instruments 684, 686, 688, 690, 692, 694, 696. A 27-point Savitzky-Golay filter has 
an approxunate corresponding frequency of about 6300 s"^ and frequencies above about 20,000 
s"^ are rapidly damped by applying this filter. In the case of a 27-point Savistzky-Golay filter, 

20 spectral data pre-processing in Figure 1 1 further mcludes applying a threshold maximum 

criterion of 1 count m the power spectrum for frequencies below 20,000 s"^ Here, data from an 
individual unit must not exhibit noise greater than 1 count at frequencies below 20,000 s"^ in 
order to satisfy the criterion. In Figure 26, the criterion is not met for units with curves 692 and 
696, since their power spectra contain points 706 and 708, each exceeding 1 count at frequencies 

25 below 20,000 s'^ The criterion is met for all other units. 

[03311 According to an alternative illustrative embodiment, a second noise criterion is applied 
instead of or m addition to the aforementioned criterion. The second criterion specifies that the 
mean power spectral intensity for a given unit be below 1 .5 counts at all frequencies. In Figure 
26, the criterion is not met for units with curves 692 and 696, since their power spectra contain 

30 points 700 and 702, each exceeding 1 .5 counts. 

[0332] The illustrative spectral data pre-processing 1 14 in Figure 1 1 and/or the factory/PM 1 10 
and pre-patient calibration 116 and correction m Figure 10 further includes applying one or more 
validation criteria to data from the factory/PM 1 10 and pre-patient 1 14 calibration tests. The 
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validation criteria identify possibly-comipted calibration data so that the data are not 
incorporated in the core classifier algorithms and/or the spectral masks of steps 132 and 130 in 
the system 100 of Figure L The validation criteria determine thresholds for acceptance of the 
results of the calibration tests. According to the illustrative embodunent, the system 1 00 of 
5 Figure 1 signals if validation criteria are not met and/or prompts retaking of the data. 
[0333] Validation includes validating the results of the factory/PM NIST 60% diflEuse 
reflectance target test 314 in Figure 10. Validation may be necessary, for example, because the 
intensity of the xenon lamp used in the test 314 oscillates during a scan over the 25-mm scan 
pattern 202 of Figure 5, The depth of modulation of measured reflected light intensity depends, 

10 for example, on the homogeneity of the illumination source at the target, as well as the collection 
efficiency over tiie scan field. The depth of modulation also depends on how well the target is 
aligned relative to the optical axis. In general, inhomogeneities of the illumination source are 
less important than inhomogeneities due to target misalignment, since illumination source 
inhomogeneities are generally accounted for by taking the ratio of reflected light intensity to 

15 incident light intensity. Thus, flie calibration 1 10, 1 16 methods use one or two metrics to sense 
oflf-center targets and prompt retaking of data. 

[0334] One such metric includes calculating a coefficient of variation, CV^), of measured 
reflected light intensity across the scan field according to Equation 48: 

^nW- /^/. A\ (48) 

20 where I(X,i) = mean [{ W(^i) - h^M l/PmQLmwian.; "std" represents standard deviation; i 

represents an interrogation point; X represents wavelength (in one embodiment, between 370 nm 
and 700 nm); and Pm(i) represents the power monitor value for hiterrogation point i. 1(3^1) is the 
mean of tiie background-subtracted (bkg), power-monitor-corrected reflectance intensity values 
&om the NIST target measured 4 times, rotating the target 90' between each measurement. 

25 Validation according to the metric of Equation 48 requires the value of CVi(X) be less than an 
experimentally-determined, fixed value, 

[0335] Another metric firom the 60% diffuse target test 3 14 includes calculating the relative 
difference, RD, between the minimum and maxunum measured intensity over the scan field 
according to Equation 49: 

30 2-[max(/».0),-mi.(/t^0U 

\am.{r(X,t)\+nMl%i)),] ^ ' 
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mean{Pm(i))f 



Here, V is scaled by the 



mean of the power monitor values. In one embodiment, the relative diflference, RD, between the 
minimum and maxunum computed in Equation 49 is more sensitive to off-centered targets than 
the coefficient of variation, CVi, computed m Equation 48. Here, validation requires the value of 
5 RD^) be less than an experhnentally-determined, fixed value. In the illustrative embodiment, 
validation requires that Equation 50 be satisfied as follows: 

RD(^) < 0.7 for X between 370 nm and 700 nm (50) 
where RD(X) is given by Equation 49. 

[0336] Validation also includes validating the results of the tungsten source test 308 jfrom 
10 Figure 1 1 using tiie approximately 99% diffuse reflectivity target. This test includes obtaining 
two sets of data, each set corresponding to a different position of the external tungsten source 
lamp. Datafiromeachsetfliatarenotaffectedby shadow are merged into one set of data. Since 
the power monitor correction is not applicable for this extemal source, a separate background 
measurement is obtained. 
15 [0337] The illustrative calibration methods 1 10, 1 1 6 use one or two metrics to validate data 
from the tungsten source test 308. One metric includes calculating a coefficient of variation, 
CVi(A,), of the mean foreground minus the mean background data, W(X,i), of the merged set of 
data, as in Equation 5 1 : 

20 where the coefficient of variation, CV^), is calculated using the mean instrument spectral 
response curve, IR, averaging over all mterrogation points of the scan pattern. Validation 
reqmres the value of CVi(X) be less than an experimentally-determmed, fixed value. In the 
illustrative embodiment, validation requires that Equation 52 be satisfied for all interrogation 
points i: 

25 CVi(X) < 0.5 for X between 370 nm and 700 nm (52) 

where CV|(X) is given by Equation 51 . 

[0338] A second metric includes calculating a mean absolute difference spectrum, MAD{X% 
comparmg the current spectral response curve to the last one measured, as in Equation 53: 

UWiX) = mea/7(|/i?,(/,^)-^M(^^)|), (53) 
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where flie instrument spectral response curve, IR, is given by Equation 39. Validation requires 
the value of MAD(A,) be less than an experimentally-determined, fixed value. In one 
embodiment, validation requires that Equation 54 be satisfied: 



5 where MAD^) is given by Equation 53. 

[0339] Validation can fijrther include validating the results of the fluorescent dye cuvette test 
306 in Figure 10, used to standardize fluorescence measurements between individual units and to 
correcting for variation in collection efficiency as a unit collects data at interrogation points of a 
scan pattern* The illustrative calibration methods 1 10, 1 16 use one or more metrics to validate 

10 data fi-om the fluorescent dye cuvette test 306 using a coefficient of variation, CViX), of dye 
cuvette intensity, Id^, as in Equation 55: 



[0340] The coefficient of variation, CV-A), in Equation 55 between about 470 nm and about 
600 nm is generally representative of fluorescence efficiency variations over the scan pattern. 

15 The coefficient of variation at about 674 imi is a measure of how well the collection system 

blocks the 337-nm excitation light. As the excitation light passes over the sxarface of the cuvette, 
the incidence and collection angles go in and out of phase, causing modulation around 574 nm. 
The coefficient of variation at about 425 imi is a measure of the cleanliness of the cuvette surface 
and is affected by the presence of fingerprints, for example. The coefficient of variation below 

20 about 400 nm and above about 700 nm is caused by a combination of the influence of 337-nm 
stray excitation light and reduced signal-to-noise ratio due to limited fluorescence firom tiie dye 
solution at these wavelengths. 

[0341] One metric includes calculating a mean coefficient of variation, CVi(X), according to 
Equation 55, between about 500 nm and about 550 nm, and comparing the mean coefficient of 
25 variation to an experimentally-determined, fixed value. According to the illustrative 
embodiment, validation requires that Equation 56 be satisfied: 



[0342] A second metric includes requiring the coefficient of variation at about 674 nm be less 
than an experimentally-determined, fixed value. In one embodiment, validation requires that 
30 Equation 57 be satisfied for all interrogation points i: 



MAD(X) < 0.2 for % between 370 nm and 700 nm 



(54) 




(55) 



mean CV^^) < 0.06 for X between 500 nm and 550 nm 



(56) 



CV,(674)<0.5 



(57) 



where CV|(A) is calculated as in Equation 55. 
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[0343] Validation can also include validating results of the fluorescent dye cuvette test 306 
using both Equations 56 and 57. Here, applying Equation 56 prevents use of data from tests 
where the scan axis is significantly shifted relative to the center of the optical axis, as well as 
tests where the cuvette is not full or is off-center. Applying Equation 57 prevents use of data 
from tests where a faulty UV emission filter is installed or where the UV filter degrades over 
time, for example. 

[0344] Validation can also include validating flie results of the 10% diffuse reflectivity custom 
target tests 3 12, 330 in Figure 10. Here, an off-center target may result in a faulty test due to 
interference at regions near the edge of the target, as well as regions near the fluorescent and 
phosphorescent plugs that are improperly masked. According to the illustrative embodiment, 
validation of the custom target tests 312, 330 requires that the relative difference between the 
minimum and maximum intensity, RD(X), is below a pre-deteimined value, where RD(3l) is 
calculated as in Equation 58: 

2-[max(/'a/)L^-min(/'a,z)L..„.J 
^ ^ lmax(7'(A./)),_,4.min(r(A,0),.«.] ^'^^ 
where (I W))*™* refers to all scan positions except those masked to avoid the plugs, as shown 
in Figures 19 and 20. In one embodiment, validation requires that Equation 59 be satisfied: 

RD^) < 1 .2 for X between 370 nm and 700 nm (59) 
where RD^) is calculated as in Equation 58. 

[0345] The invention can also validate the results of the null target test 304, 328 in Figure 1 0. 
The null target test is used, for example, to account for internal stray light in a given instrument 
According to the illustrative embodiment, a maximum allowable overall amount of stray light is 
unposed. For example, in one preferred embodiment, validation of a null target test 304, 328 
requires the integrated energy, DB, be below a predetermined value, where IE is calculated fiom 
background-subtracted, power-monitor-corrected null target reflectance intensity measurements, 
as in Equation 60: 




nuUiA,i)-bkg(A,i) 



(60) 




where AX in the summation above is about 1-nm, In one embodiment, validation requires that 
Equation 6 1 be satisfied: 



IE < 4000 counts 



(61) 
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vAi&ce IE is calculated as in Equation 60. 

[0346] The invention may also employ validation of the open air target test 3 1 0 in Figure 1 0. 
Like the null target test 304, 328, the open air target test is used in accounting for internal stray 
light in a given instrument. Accordrag to the illustrative embodiment, validation of an open air 
5 target test 3 10 requires the integrated energy, IE, be below a predetermined value, where IE is 
calculated as in Equation 60, except using open air reflectance intensity measurements in place 
of null target measurements, null(X4). By way of example, in one case validation requires that 
the value of integmted energy for the open air test be below 1 .2 times the integrated energy from 
the null target test, calculated as in Equation 60. 

1 0 [0347] According to another feature, the invention validates the power monitor corrections 
used in the calibration tests in Figure 10. Patient and calibration data that use a power monitor 
correction may be erroneous if the illumination source misfires. According to one approach, 
validation of a power monitor correction requires that the maximum raw power monitor intensity 
TGddmgf Pm mnCiX be greater than a predetermined minimum value and/or be less than a 

1 5 predetermined maximum value at each interrogation point i. In the illustrative embodiment, 
validation requires that Equation 62 be satisfied: 

6000 counts < < 30,000 counts for all i (62) 

[0348] According to the illustrative embodiment, spectral data pre-processing 1 14 in Figure 1 1 
includes accoimting for the result of the real-time motion tracker 106 in the system 100 of Figure 

20 1 when applying the correction factors in block diagram 340 of Figure 1 1 . As discussed herein, 
the system 100 of Figure 1 applies the calibration-based corrections in Figure 1 1 to spectral data 
acquired from a patient scan. These corrections are applied by matching spectral data from each 
interrogation point in a patient scan to calibration data from a corresponding interrogation point. 
However, a patient scan of the 499 mterrogation points shown in the scan patten 202 of Figure 5 

25 takes approximately 12 seconds. During those 12 seconds, it is possible that the tissue will shift 
slightly, due to patient movement. Thus, spectral data obtamed during a scan may not 
correspond to an initial index location, since the tissue has moved from its original position in 
relation to the scan pattern 202. The real-time motion tracker 106 of Figure 1 accounts for this 
movement by using data from video images of the tissue to calculate, as a ftinction of scan time, 

30 a translational shift in terms of an x-displacement and a y-displacement. The motion tracker 106 
also validates the result by determining whether the calculated x,y translational shift accurately 
accoimts for movement of the tissue in relation to the scan pattern or some other fixed standard 
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such as fhe initial position of component(s) of the data acquisition syst^ (the camera and/or 
spectroscope). The motion tracker 106 is discussed in more detail below. 
[0349] Illustratively, the spectral data pre-processing 1 14 in Figure 1 1 accounts for the result 
of the real-time motion tracker 106 by applying a calibration spectra lookup metiiod. The lookup 

5 method includes obtaining the motion-corrected x,y coordinates corresponding to the position of 
the center of an interrogation point firom which patient spectral data is obtained during a patient 
scan. Then tiie lookup method includes using the x,y coordinates to find tiie calibration data 
obtained firom an interrogation point whose center is closest to the x,y coordinates. 
[0350] The scan pattern 202 of Figure 5 is a regular hexagonal sampling grid with a pitch 

10 (center-to-center distance) of 1 . 1 mm and a maximum interrogation point spot size of 1 mm. 
This center-to-center geometry indicates a horizontal pitch of 1 .1 mm, a vertical pitch of about 
0.9527 mm, and a maximum comer distance of the circumscribed regular hexagon to the center 
of 0.635 mm. Thus, the illustrative lookup method finds the calibration interrogation point 
whose center is closest to the motion-corrected x,y coordinates of a patient scan interrogation 

IS point by finding coordmates of a calibration point that is less than 0.635 mm firom x,y. 

[0351] The background spectra, BkgndQ, in Figure 1 1, are obtained at nearly the same time 
patient spectral data are obtained and no motion correction factor is needed to backgroxmd- 
subtract patient spectral data. For example, at a given interrogation point during a patient scan, 
the system 100 of Figure 1 pulses the UV light source on only while obtaining fluorescence data, 

20 then pulses the BB 1 light source on only while obtaining the first set of reflectance data, then 
pulses the BB2 light source on only while obtaining the second set of reflectance data, then 
obtains the background data, Bl^dQ, at the interrogation point with all internal light sources 
off. All of tius data is considered to be approximately simultaneous and no motion correction 
Victor is needed for the Bl^dQ calibration data. 

25 [0352] The real-time motion tracker 106 of Figure 1 uses video data obtained firom the tissue 
contemporaneously with the spectral data. In addition to motion correction, the system of Figure 
1 uses video (image) data to determine image masks for disease probability computation, to 
focus the probe 142 through which spectral and/or image data is acquired, and to compute a 
brightness and contrast correction and/or image enhancement for use in disease overlay display. 

30 Patient scan procedure 

[0353] Figure 27A is a block diagram 714 showing steps an opemtor performs before a patient 
scan as part of spectral data acquisition 104 in the system 100 of Figure 1, according to an 
illustrative embodiment of the invention. The steps in Figure 27A are arranged sequentially with 
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respect to a time axis 716. As shown, an operator applies a contrast agent to the tissue sample 
718, marks the time application is complete 720, focuses the probe 142 through which spectral 
and/or image data will be obtained 722, then initiates the spectral scan of the tissue 724 within a 
pre-determined window of time. 

5 [0354] According to the illustrative embodiment, the window of time is an optimum range of 
tune following application of contrast agent to tissue within which an approximately 12 to 15 
second scan can be peiformed to obtain spectral data that are used to classify tissue samples wiHi 
a high degree of sensitivity and selectivity. The optimum window should be long enough to 
adequately allow for restarts indicated by focusing problems or patient movement, but short 

10 enough so that the data obtained is consistent. Consistency of test data is needed so that tissue 
classification results for the test data are accurate and so that the test data may be added to a bank 
of reference data used by the tissue classification scheme. In one illustmtive embodiment, the 
optimum window is expressed in terms of a fixed quantity of time following application of 
contrast agent. In another illustmtive embodiment, the optimum window is expressed in terms of 

IS a threshold or range of a trigger signal fi'om the tissue, such as a reflectance intensity indicative 
of degree of tissue whiteness. 

[0355] The contrast agent in Figure 27A is a solution of acetic acid. According to one 
exemplary embodiment, the contrast agent is a solution between about 3 voliune percent and 
about 6 volume percent acetic acid in water. More particularly, in one preferred embodiment, 
20 the contrast agent is an about 5 volume percent solution of acetic acid in water. Other contrast 
agents may be used, including, for example, formic acid, propionic acid, butyric acid, Lugol's 
iodine, Shiller's iodine, methylene blue, toluidine blue, indigo carmine, indocyanine green, 
fluorescein, and combinations of tiiese agents. 

[0356] According to the illustrative embodiment, the time required to obtain results firom a 
25 patient scan, following pre-patient calibration procedures, is a maximimi of about 5 minutes. 

Thus, in Figure 27A, the five-minute-or-less procedure includes applying acetic acid to the tissue 
sample 726; focusing the probe (142) 728; waiting, if necessary, for the beguxning of the 
optimum pre-determined window of time for obtaining spectral data 730; obtaining spectral data 
at all interrogation points of the tissue sample 732; and processing the data using a tissue 
30 classification scheme to obtain a diagnostic display 734. The display shows, for example, a 
reference image of the tissue sample with an overlay indicating regions that are classified as 
necrotic tissue, indetermmate regions, healthy tissue (no evidence of disease, NED), and CIN 2/3 
tissue, thereby indicating where biopsy may be needed. 



wo 2004/095359 



PCTAJS2004/011820 



.73. 

[0357] The times indicated in Figure 27A may vary. For example, if the real-time motion 
tracker 106 in the system of Figure 1 indicates too much movement occurred during a scan 732, 
the scan 732 may be repeated if there is sufiBcient time left in the optimum window. 
[0358] Figure 27B is a block diagram 738 showing a time line for the spectral scan 732 

S indicated in Figure 27A. In the embodiment shown in Figure 27B, a scan of all interrogation 
points of the scan pattern (for example, the scan pattern 202 of Figure 5) takes from about 12 
seconds to about 1 S seconds, during which time a sequence of unages is obtained for motion 
tracking, as perforaied in step 106 of the system 100 of Figure 1. By the time a scan begins, a 
motion-tracking starting image 742 and a target laser image 744 have been obtained 740. The 

10 target laser image 744 may be used for purposes of off-line focus evaluation, for example. 

During the acquisition of spectral data during the scan, a frame grabber 120 (Figure 1) obtains a 
single image about once every second 746 for use in monitoring and/or correcting for movement 
of the tissue from one frame to the next. In Figure 27B, a frame grabber acquires images 748, 
750, 752, 754, 756, 758, 760, 762, 764, 766, 768 that are used to track motion that occurs during 

15 the scan. 

[0359] Image data from a video subsystem is used, for example, in target focusing 728 in 
Figure 27A and in motion tracking 106, 746 in Figure 27B. Image data is also used in detecting 
the proper alignment of a target in a calibration procedure, as well as detecting whether a 
disposable is in place prior to contact of the probe with a patient. Additionally, in one 
20 embodiment, colposcopic video allows a user to monitor the tissue sample throughout the 
procedure. 

Video calibration and focusing 
[0360] Figure 28 is a block distrain 770 that shows the architecture of an illustrative video 
subsystem used in the system 100 of Figure 1. Figure 28 shows elements of the video subsystem 

25 in relation to components of the system 100 of Figure 1 . The video subsystem 770 acquires 

single video images and real-time (streaming) video images. The video subsystem 770 can post- 
process acquired image data by applying a mask overlay and/or by adding other graphical 
annotations to the acquired image data. Illustratively, image data is acquired in two frame 
buffers during real-time video acquisition so that data acquisition and data processing can be 

30 alternated between buffers. The camera(s) 772 in the video subsystem 770 of Figure 28 include 
a camera located in or near the probe head 192 shown in Figure 4, and optionally includes a 
colposcope camera extemal to the probe 142 for visual monitormg of the tissue sample during 
testing. In one illustrative embodunent, only the probe head camera is used. Figure 28 shows a 



wo 2004/095359 



PCTAJS2004/011820 



-74- 

haidware interfece 774 between the cameras 772 and the rest of the video subsystem 770. The 
frame grabber 120 shown in Figure 1 acquires video data for processing in other components of . 
the tissue characterization system 100. In one embodiment, the frame grabber 120 uses a card 
for video data digitization (video capture) and a card for broadband illumination (for example, 
5 flash lamps) control. For example, one embodiment uses a Matrox Meteor 2 card for digitization 
and an Imagenation PXC-200F card for illumination control, as shovra in block 776 of Figure 28. 
[0361] Real-time (streaming) video images are used for focusmg the probe optics 778 as well 
as for visual colposcopic monitoring of the patient 780. Single video images provide data for 
calibration 782, motion tracking 784, image mask computation (used in tissue classification) 
10 786, and, optionally, detection of the presence of a disposable 788. In some illustrative 

embodiments, a smgle reference video image of the tissue sample is used to compute the image 
masks 108 in the system 100 of Figure 1. This reference image is also used in detennining a 
brightness and contrast correction and/or other visual enhancement 126, and is used in the 
disease overlay display 138 m Figure 1. 

15 [0362] The illustrative video subsystem 770 acquires video data 790 from a single video image 
witiiin about 0.5 seconds. The video subsystem 770 acquires single images in 24-bit RGB 
format and is able to convert Aem to grayscale images. For example, image mask computation 
108 in Figure 1 converts fbs RGB color triplet data into a single luminance value, Y, (grayscale 
intensity value) at each pixel, where Y is given by Equation 63: 

20 Y = 0.299R + 0.587G + 0. 1 14B (63) 

where the grayscale intensity component, Y, is expressed m terms of red (R), green (G), and blue 
(B) mtensities; and where R, G, and B range from 0 to 255 for a 24-bit RGB image. 
[0363] Laser target focusing 728 is part of the scan procedure m Figure 27A. An operator uses 
a targeting laser in conjunction wifli real-time video to quickly align and focus the probe 142 

25 prior to startmg a patient scan. In the illustrative embodiment, an operator performs a laser 

"spot" focusing procedure in step 728 of Figure 27A ^ere the operator adjusts the probe 142 to 
align laser spots projected onto the tissue sample. The user adjusts the probe while looking at a 
viewfinder with an overlay indicating the proper position of the laser spots. In one alternative 
embodiment, an operator instead performs a thin-line laser focusing method, where the operator 

30 adjusts the probe until the laser lines become sufficiently thin. The spot focus method eHiaws for 
faster, more accurate focusing than a line-width-based focusing procedure, since thin laser lines 
can be difficult to detect on tissue, particularly dark tissue or tissue obscured by blood. Quick 
focusing is needed in order to obtain a scan withm the optimal time wmdow following 
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application of contrast agent to the tissue; thus, a spot-based laser focusing method is preferable 
to a thin line method, although a thin line focus method may be used in alternative embodiments. 
[0364] A target focus validation procedure 122 is part of the tissue characterization system 100 
of Figure 1, and determines whether the optical system of the instrument 102 is in focus prior to 
a patient scan. If the system is not in proper focus, the acquired fluorescence and reflectance 
spectra may be erroneous. Achieving proper focus is important to the integrity of the image 
masking 108, real-time tracking 106, and overall tissue classification 132 components of the 
system 100 of Figure 1. 

[0365] The focus system includes one or more target laser(s) that project laser light onto the 
patient sample prior to a scan. In one embodiment, the targeting laser(s) project laser light from 
the probe head 192 toward the sample at a slight angle with respect to the optical axis of the 
probe 142 so that the laser light that strikes the sample moves within the image frame when the 
probe is moved with respect to the focal plane. For example, in one illustrative embodiment, 
four laser spots are directed onto a target such that ^en the probe 142 moves toward Ae target 
during focusing, the spots move closer together, toward the center of the image. Similarly, when 
the probe 142 moves away from the target, the spots move further apart within the image frame, 
to^^d the comers of the image. 

[0366] Figure 29A is a smgle video image 794 of a target 796 of 10% diffuse reflectivity upon 
which a target laser projects a focusing pattern of four laser spots 798, 800, 802, 804. During 
laser target focusing 728 (Figure 27A), an operator views four focus rings that are displayed at 
predetermined locations, superimposed on the target focusing image. Figure 29B depicts the 
focusing image 794 on the target 796 in Figure 29A with superimposed focus rings 806, 808, 
810, 812. The operator visually examines the relative positions of the laser spots 798, 800, 802, 
804 in relation to the corresponding focus rings 806, 808, 810, 8 12 while moving the probe head 
192 along the optical axis toward or away from the target/tissue sample. When flie laser spots lie 
withm the focus rings as shown in Figure 29B, the system is within its required focus range. The 
best focus is achieved by aligning the centers of all the laser spots with the corresponding centers 
of the focus rings. Alternatively, spot patterns of one, two, three, five, or more laser spots may 
be used for focus alignment. 

[0367] It is generally more difficult to align laser spots that strike a non-flat tissue sample 
target than to align the spots on a flat, uniform target as shown in Figure 29B. In some instances, 
a laser q)ot projected onto tissue is unclear, mdistinct, or invisible. Visual evaluation of focus 
may be subjective and qualitative. Thus, a target focus validation procedure is usefiil to insure 
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proper focus of a tissue target is achieved. Proper focus allows the comparison of both image 
data and spectral data from different instrument imits and different operators. 
[0368] In one illustrative embodiment, the system 100 of Figure 1 performs an automatic 
target focus validation procedure usmg a smgle focus image. The focus hnage is a 24-bit RGB 
5 color image that is obtained before acquisition of spectral data in a patient scan. The focus 
image is obtained with the targeting laser turned on and the broadband lights (white lights) 
turned off. Automatic target focus validation mcludes detecting the locations of the centers of 
visible laser spots and measuring their positions relative to stored, calibrated positions 
("nominal" center positions). Then, the validation procedure applies a decision rule based on the 
10 number of visible laser spots and their positions and decides whether the system is in focus and a 
spectral scan can be started. 

[0369] Figure 3 0 is a block diagram 8 1 6 of a target focus validation procedure according to an 
illustrative embodiment of the invention. The steps include obtaining a 24-bit RGB focus image 
818, performing image enhancement 820 to highlight the coloration of the laser spots, 

15 performmg morphological image processing (dilation) to fill holes and gaps within the spots 822, 
defining a region of interest (ROI) of the image 824, and computing a mean and standard 
deviation 826 of the luminance values (brightness) of pixels within the region of interest. Next, 
the focus validation procedure iteratively and dynamically thresholds 828 the enhanced focus 
image using the computed mean and standard deviation to extract the laser spots. Between 

20 thresholding iterations, morphological processing 830 disconnects differentiated image objects 
and removes small image objects from the thresholded binary image, while a region analysis 
procedure 832 identifies and removes image objects located outside the bounds of the target laser 
spot pathways 838 and objects whose size and/or shapt do not correspond to a target laser spot. 
After all thresholding iterations, tiie found "spots" are either verified as true target laser spots or 

25 are removed firom the image 834, based on size, shape, and/or location. Next, in step 842, the 
focus validation procedure computes how far the centers of the found spots are firom the nominal 
focus centers and converts the difference from pixels to millimeters in step 844. The validation 
procedure then applies a decision rule based on the number of found spots and their positions 
and decides whether the system is in focus such that a spectral scan of the patient can begin. 

30 [0370] The focus validation procedure of Figure 30 be^ with obtaining the 24-bit RGB 
focus image and splitting it into R, G, and B channels. Each channel has a value in the range of 
0 to 255. Figure 3 1 depicts the RGB focus image 794 firom Figure 29A with certain illustrative 
geometry superimposed. Figure 31 shows the four nominal spot focus centers 850, 852, 854, 
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856 as red dots, one of which is the red dot labeled **N" in quadrant 1. The nominal spot focus 
centers represent the ideal location of centers of the projected laser spots, achieved when the 
probe optics are in optimum focus. The nominal spot focus centeis 850, 852, 854, 856 
correspond to the centers of the rings 806, 808, 810, 812 in Figure 29B. An (x,y) position is 
5 determined for each nominal focus center. A nominal image focus center (857), O, is defined by 
the mtersection of the two red diagonal Imes 858, 860 in Figure 31. The red diagonal lines 858, 
860 connect the two pairs of nominal spot focus centers 852, 854 in quadrants 2 and 3 and 850, 
856 in quadrants 1 and 4, respectively. Also, the slopes of the two lines 858, 860 are computed 
for later use. 

10 10371] Step 820 in the procedure of Figure 30 is image enhancement to highlight the 

coloration of the laser spots in contrast to the surrounding tissue. In one embodiment, the R 
value of saturated spots is '"red clipped" such that if R is greater than 180 at any pixel, the R 
value is reduced by 50. Then, a measure of greenness, Ge, of each pixel is computed as in 
Equation 64: 

15 Ge = G-R-15 (64) 

where G is the green value of a pixel, R is the red value of the pixel, and 1 5 is a correction factor 
to remove low intensity noise, experimentally-determined here to be 15 gray levels. 
[0372] Figure 32A represents the green channel of an RGB image 864 of a cervical tissue 
sample, used in an exemplary target focus validation procedure. In this image, only two top 

20 focus laser spots 868, 870 are clear. The lower right spot 872 is blurred/difiused while the lower 
left spot 874 is obscured. The green-channel luminance (brigjitness), Ge, of the green-enhanced 
RGB image 864 of Figure 32A may be computed using Equation 64 and may be displayed, for 
example, as grayscale luminance values between 0 and 255 at each pixel. 
[0373] In step 822 of Figure 30, the focus validation procedure performs morphological 

25 dilation using a 3x3 square structuring element to fill holes and gaps within the found spots. 
Then m step 824, the procedure uses a pre-defined, cuxjular region of interest (ROI) for 
computmg a mean, M, and a standard deviation, STD, 826 of the greenness value, Ge, of the 
pixels within the ROI, which are used in iterative dynamic thresholding 828. According to the 
illustrative embodiment, the ROI is a substantially circular region with a 460-pixel diameter 

30 whose center coincides with the nominal image focus center, O. 

[0374] Before iterative dynamic thresholding begins, Ge is set equal to zero at a 50-pixel 
diameter border about the ROI. Then, iterative dynamic thresholding 828 begins by setting an 
iteration variable, p, to zero, then computing a threshold value, Th, as follows: 
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Th = M + pSTD (65) 
where M and STD are defined firom the ROI. Since p=0 m the first iteration, the threshold, Th, is 
a "mean'* greenness value over the entire ROI in the first iteration. In this embodiment, image 
thresholding is a subclass of image segmentation that divides an image into two segments. The 
5 result is a binary image made up of pixels, each pixel having a value of either 0 (off) or 1 (on). 
In step 828 of the focus validation procedure of Figure 30, the enhanced greenness value of a 
pixel corresponding to point (x,y), within the ROI, GE(x,y), is compared to the threshold value, 
Th. The threshold is applied as in Equation 66: 

IF GE(x,y) > Th, THEN the binary pixel value at (x,y), Bt = 1, else Bt = 0. (66) 

10 [0375] Iterative dynamic thresholding 828 proceeds by performing morphological opening 830 
to separate nearby distinguishable image objects and to remove small objects of the newly 
thresholded binary image. According to the illustrative embodiment, the morphological opening 
830 includes performing an erosion, followed by a dilation, each using a 3x3 square structuring 
element. The procedure then determines the centroid of each of Ihe thresholded objects and 

15 removes each object whose center is outside the diagonal bands bounded by two lines that are 40 
pixels above and below the diagonal lines 858, 860 in Figure 3 1 . These diagonal bands include 
the region between lines 876, 878 and the region between lines 880, 882 in Figure 31, 
determined in step 838 of Figure 30. An image object whose center lies outside these bands does 
not correspond to a target focus spot, since the centers of the focus laser spots should appear 

20 within these bands at any position of the probe along the optical axis. The spots move closer 
together, within the bands, as the probe moves closer to the tissue sample, and the spots move 
farther apart, within the bands, as the probe moves away from the tissue sample. 
[0376] Next, step 832 of the thresholding iteration 828 computes an area (A), eccentricity (E), 
and equivalent diameter (ED) of the found image objects, and removes an object whose size 

25 and/or shape - described here by A, E, and ED - does not correspond to that of a focus laser 
spot. Band ED are defined as follows: 

E = (l-bW)^'^ (67) 
ED = 2(A/nf^ (68) 
where a is the minor axis length and b is the major axis length in units of pixels. For example, 
30 step 832 applies Equation 69 as follows: 

IF A > 5000 OR IF E> 0.99 OR IF ED > 110, 

THEN remove object (set Bt = 0 for all pixels in object). (69) 
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Other criteria may be applied. For example, Equation 70 may be applied in place of Equation 
69: 

IF A > 2500 OR IF E > 0.99 OR IF ED > 80, 

THEN remove object (set Bt = 0 for all pixels in object). (70) 
5 [0377] Next, the iteration variable, p, is increased by a fixed value, for example, by 0.8, and a 
new threshold is calculated using Equation 65. The iteration proceeds by applying the new 
threshold, performing a morphological opening, computing centroids of the newly thresholded 
regions, removing regions whose center position, size, and/or shape do not correspond to those 
of a target focus spot, and stepping up the value of the iteration variable p. Iterative dynamic 
10 thresholding proceeds until a pre-determined condition is satisfied. For example, the 
thresholding ends when the following condition is satisfied: 

IF p > 6 OR F the number of qualified spots (image objects) < 4, THEN STOP. (71) 

[0378] Step 834 of the focus validation procedure eliminates any image object remaining after 
dynamic thresholding that does not meet certain laser spot size and shape criteria. For example, 
15 according to the illustrative embodiment, step 834 applies the condition in Equation 72 for each 
remaining image object 

IF A < 80 OR IF E> 0.85 OR IF ED < 10, THEN remove object. (72) 
[0379] In an alternative embodiment, one or more additional criteria based on the position of 
each imj^e object (found spot) are applied to eliminate olgects that axe still within tiie focus 
20 bands of Figure 3 1 , but are too fer firom the nommal centers 850, 852, 854, 856 to be valid focus 
spots. 

[0380] Figure 32B shows an image 898 of the cervical tissue sample of Figure 32A following 
step 834, wherein the top two image objects were verified as target laser spots, while the bottom 
objects were eliminated. 

25 [0381] Step 842 ofthe focus validation procedure assigns each ofthe found spots to its 

respective quadrant and computes Ihe centroid of each found spot Figure 3 1 shows the found 
spots as blue dots 900, 902, 904, 906. Then for each found spot, step 842 computes the distance 
between the center of the spot to the nominal image focus center 857, 0. For the focus q)ot 
center 900 labeled 'T" in Figure 31, this distance is Lof, tiie length ofthe blue line 910 fiom 

30 point O to point F. The distance between the nominal focus center, N, 850 corresponding to the 
quadrant containing the found spot, and the nominal unage focus center 857, 0, is Lqn, the 
length ofthe red line 912 fi-om point O to point N. Step 842 ofthe focus validation procedure 
then determines a focus value for verified focus spot 900 equal to the diffwence between the 



wo 2004/095359 



PCT/US2004/011820 



-80- 

lengths LoF and Low. The focus value of each of the verified focus spots is computed in this 
manner, and the focus values are converted from pixels to millimeters along the focus axis (z- 
axis) in step 844 of Figure 30 using an empirically-determined conversion ratio - for example, 
0,34 mm per pixel. 

[0382] Next, the focus validation procedure of Figure 30 applies a decision rule in step 846 
based on the numb^ of found spots and their positions. The decision rule is a quantitative 
means of deciding whether the system is in focus and a spectral scan of the tissue can begin. 
According to the illustrative embodunent, step 846 applies a decision rule given by Equations 
73, 74, and 75: 

IF 3 or more spots are foimd, THEN (73) 
IF the focus value determined in step 842 is < 6 mm for any 3 spots OR 
IF the focus value is < 4 mm for any 2 spots, 
THEN "Pass", ELSE "FaU'' (require refocus). 



IF only 2 spots are found, THEN 
IF the focus value of any spot is > 4 nun, 
THEN "Fail" (require refocus), ELSE "Pass". 



(74) 
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ip < 1 spot is found, THEN (75 
"Fail" (require refocus). 

Other decision rules may be used alternatively. 

[0383J Figures 33 and 34 show the application of the focus validation procedure of Figure 30 
using a rubber cervix model placed so that the two upper laser spots are within the os region. 
For this example, the distance between the edge of the probe head 192 and the target (or target 
tissue) is approximately 100 mm at optimum focus, and the distance light travels between the 
target (or target tissue) and the first optic within the probe 142 is approximately 130 mm at 
optimum focus. 

[0384] Figure 33 is a 24-bit RGB target laser focus image 942 of a rubber cervbc model 944 
onto which four laser spots 946, 948, 950, 952 are projected. The cervix model 944 is off-center 
in the im^e 942 such that tiie two upper laser spots 946, 948 lie within the os region. Figure 34 
shows a graph 954 depicting as a function of probe position relative to the target tissue 956, the 
mean of a focus value 958 (in pixels) of each of the four laser spots 946, 948, 950, 952 projected 
onto the rubber cervix model 944. The curve fit 960 of tiie data indicates tiie relationship 
between measured focus, f, 958 and probe location, Zp, 956 (in mm) is substantially linear. 
However, tiie curve is shifted down and is not centered at (0,0). This indicates a focus error 
introduced by the manual alignment used to obtain the 2=0 focus position. Such an enor may 
ptonqrt a "Fail" determination in step 846 of the focus validation procedure of Figure 30, 
depending on the chosen decision rule. Figure 34 indicates the difiBculty m making a visual 
focus judgment to balance the focus of the four spots, particularly where the target surface 
(tissue sample) is not flat and perpendicular to the optical axis (z-axis) of the probe system. 
[0385] The focus validation procedure illustrated in Figure 30 provides an automatic, 
quantitative check of the quality of focus. Additionally, in flie illustiative embodiment, the focus 
validation procedure predicts the position of optimum focus and/or automatically focuses the 
optical system accordingly by, for «cample, triggering a galvanometer subsystem to move the 
probe to the predicted position of optimum focus. 

[0386] The focus validation procedure in Figure 30 produces a final decision m step 846 of 
"Pass" or "Fail" for a given focus image, based on tiie decision rule given by Equations 73-75. 
This indicates whether the focus achieved for this tissue sample is satisfactory and whether a 
spectral data scan may proceed as shown m step 732 of Figures 27A and 27B. 
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Determining optimal data acquisition window 
[03871 After application of contrast agent 726 and target focusing 728, step 730 of Figure 27A 
indicates that the operator waits for the beginning of the optimum window for obtaining spectral 
data unless the elapsed time already exceeds the start of the window. The optimum window 
5 indicates the best time period for obtaining spectral data, following application of contrast agent 
to the tissue, considering the general time constraints of the entire scan process in a given 
embodiment. For example, according to the illustrative embodiment, it takes from about 12 to 
about 15 seconds to perforai a spectral scan of 499 interrogation points of a tissue sample. An 
opthnum window is determined such that data may be obtained over a span of time within this 

10 window from a sufficient number of tissue regions to provide an adequately detailed indication 
of disease state with sufficient sensitivity and selectivity. The optimum window preferably, also 
allows the test data to be used, in tum, as reference data in a subsequently developed tissue 
classification module. According to another feature, the optimum window is wide enough to 
allow for restarts necessitated, for example, by focusing problems or patient movement. Data 

15 obtained wifhm the optimum window can be added to a bank of reference data used by a tissue 
classifiication scheme, such as component 132 of the system 100 of Figure 1. Thus, the optimum 
window is preferably narrow enough so that data from a given region is sufficiently consistent 
regardless of when, within the optimum window, it is obtained. 

[0388] According to the illustrative embodiment, the optimal window for obtaining spectral 
20 data m step 1 04 of Figure 1 is a period of tune from about 30 seconds following application of 
the contrast agent to about 130 seconds following application of the contrast agent. The time it 
takes an operator to apply contrast agent to the tissue sample may vary, but is preferably between 
about 5 seconds and about 1 0 seconds. The operator creates a time stamp in the illustrative scan 
procedure of Figure 27A after completing application of the contrast agent, and then waits 30 
25 seconds before a scan may begin, where the optimum window is between about 30 seconds and 
about 130 seconds following application of contrast agent. If the scan takes from about 12 
seconds to about 15 seconds to complete (where no retake is required), the start of the scan 
procedure must begin soon enough to allow all the data to be obtained within the optimum 
window. In other words, m this embodiment, the scan must begin at least before 1 15 (assuming 
30 a worst case of 1 5 seconds to complete the scan) seconds following the time stamp (115 seconds 
after application of contrast agent) so that the scan is completed by 130 seconds following 
application of contrast agent Other optimum windows may be used. In one embodiment, the 
optimum window is between about 30 seconds and about 1 10 seconds following application of 
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contrast agent. One alternative embodiment has an optimal window with a "start" thne from 
about 10 to about 60 seconds following application of acetic acid, and an "end" time from about 
1 1 0 to about 180 seconds following application of acetic acid. Other optimum windows may be 
used. 

5 [0389] In one illustrative embodiment, the tissue characterization system 100 of Figure 1 
includes identifying an optimal window for a given application, and/or subsequently using 
spectral data obtained within the pre-detemiined window in a tissue classification module, such 
as step 132 of Figure 1. According to one feature, optimal windows are determined by obtaining 
optical signals from reference tissue samples with known states of health at various times 

10 following application of a contrast agent. 

[0390] Determining an optimal window illustratively includes the steps of obtaining a first set 
of optical signals from tissue samples having a known disease state, such as CIN 2/3 (grades 2 
and/or 3 cervical intraepithelial neoplasia); obtaining a second set of optical signals from tissue 
samples having a different state of health, such as non-CIN 2/3; and categorizing each optical 

15 signal mto *1)ms" according to the time it was obtained in relation to the time of application of 
contrast agent The optical signal may include, for example, a reflectance spectrum, a 
fluorescence spectrum, a video image mtensity signal, or any combination of these. 
[0391] A measure of the difference between the optical signals associated wilh the two types 
of tissue is then obtained, for example, by determining a mean signal as a ftinction of wavelength 

20 for each of the two types of tissue samples for each time bin, and using a discrimination fimction 
to determine a weighted measure of difference between the two mean optical signals obtained 
within a given time bin. This provides a measure of the difference between the mean optical 
signals of the two categories of tissue samples - diseased and healthy - weighted by the variance 
between optical signals of samples withm each of the two categories. 

25 [0392] According to the illustrative embodunent, the invention fiirther includes developing a 
classification model for each time bin for the purpose of determining an optimal window for 
obtaining spectral data in step 104 of Figure 1. After determining a measure of difference 
between the tissue types in each bin, an optimal window of time for diflferentiating between 
tissue types is determined by identifying at least one bm in which the measure of difference 

30 between the two tissue types is substantially maximized. For example, an optimal window of 
time may be chosen to include every time bin in which a respective classification model provides 
an accuracy of 70% or greater. Here, the optimal window describes a period of time following 
application of a contrast agent in which an optical signal can be obtained for purposes of 
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classifying the state of health of the tissue sample with an accuracy of at least 70%. Models 
distinguishing between three or more categories of tissue may also be used in determining an 
optimal window for obtaining spectral data. As discussed below, other factors may also be 
considered hi determining the optunal window. 
5 [0393] An analogous embodiment includes determining an optimal threshold or range of a 
measure of change of an optical signal to use in obtaining (or triggering the acquisition of) the 
same or a different signal for predicting flie state of healfli of the sample. Instead of determming 
a specific, fixed wmdow of time, this embodiment includes determming an optimal threshold of 
change in a signal, such as a video image whiteness intensity signal, after which an optical 

10 signal, such as a diffuse reflectance spectrum and/or a fluorescence spectrum, can be obtained to 
accurately characterize the state of health or other characteristic of the sample* This illustrative 
embodiment includes monitoring reflectance and/or fluorescence at a single or multiple 
wavelength(s), and upon reaching a threshold change from the initial condition, obtaining a full 
reflectance and/or fluorescence spectrum for use in diagnosing the region of tissue. This method 

1 5 allows for reduced data retrieval and monitoring, smce it involves continuous tracking of a 
single, partial-spectrum or discrete-wavelength ^trigger" signal (instead of multiple, full- 
spectrum scans), followed by the acquisition of spectral data in a spectral scan for use in tissue 
characterization, for example, the tissue classification module 132 of Figure 1. Alternatively, the 
trigger may include more than one discrete-wavelength or partial-spectrum signal. The measiure 

20 of change used to trigger obtaining one or more optical signals for tissue classification may be a 
weighted measure, and/or it may be a combination of measures of change of niore than one 
signal. 

[0394] In a further illustrative embodiment, instead of determining an optimal threshold or 
range of a measure of change of an optical signal, an optimal threshold or range of a measure of 

25 the rate of change of an optical signal is determined. For example, the rate of change of 
reflectance and/or fluorescence is monitored at a single or multiple wavelength(s), and upon 
reaching a threshold rate of change, a spectral scan is performed to provide spectral data for use 
in diagnosing the region of tissue. The measure of rate of change used to trigger obtaining one 
or more optical signals for tissue classification may be a weighted measure, and/or it may be a 

30 combination of measures of change of more than one signal. For example, the measured rate of 
change may be weighted by an initial signal intensity. 

[0395] According to the illustrative embodiment, the optimum time window includes a time 
window in which spectra firom cervical tissue may be obtained such that sites indicative of 
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grades 2 and 3 cervical intraepiflielial neoplasia (CIN 2/3) can be separated from non-CIN 2/3 
sites. Non-CIN 2/3 sites include sites with grade 1 cervical intraepithelial neoplasia (CIN 1), as 
well as NED sites, normal columnar and normal squamous epithelia, and mature and immature 
metaplasia. Altemately, sites indicative of high grade disease, CIN 2+, which includes CIN 2/3 
5 categories, carcinoma in situ (CIS), and cancer, may be separated from non-higji-grade-disease 
sites. In general, for any embodiment discussed herein in which CIN 2/3 is used as a category 
for classification or characterization of tissue, the more expansive category CIN 2+ may be used 
altematively. Preferably, the system 100 can differentiate amongst three or more classification 
categories. Exemplary embodiments are described below and include analysis of the time 

10 response of diflftise reflectance and/or 337-nm fluorescence spectra of a set of reference tissue 
samples with regions having known states of health to determme temporal characteristics 
mdicative of the respective states of health. These characteristics are then used in building a 
model to determine a state of health of an unknown tissue sample. Other illustrative 
embodiments include analysis of fluorescence spectra using other excitation wavelengths, such 

15 as 380 nm and 460 nm, for example. 

[0396] According to one illustrative embodiment, an optimum window is deteraiined by 
tracking the difference between spectral data of two tissue types using a discrimmation function. 
[0397] According to the illustrative embodhnent, the discrimination function shown below in 
Equation 76 may be used to extract differences between tissue types: 

where |i corresponds to the mean optical signal for the tissue type indicated m the subscript; and 
a corresponds to the standard deviation. The categories CIN 2/3 and non-CIN 2/3 are used in 
this embodiment because spectral data is particularly well-suited for differentiating between 
these two categories of tissue, and because spectral data is prominently used in one embodiment 

25 of the classification schema ui the tissue classification module in step 1 32 of Figure 1 to identify 
CIN 2/3 tissue. Thus, in this way, it is possible to tailor the choice of an optimal scan v^dow 
such that spectral data obtained within that window are well-adapted for use in identifying CIN 
2/3 tissue in the tissue classification scheme 132. In one illustrative embodiment, the optical 
signal in Equation 76 includes diffuse reflectance. In another illustrative embodiment, the 

30 optical signal includes 337-nm fluorescence emission spectra. Other illustrative embodiments 
use fluorescence emission spectra at another excitation wavelengfli such as 380 nm and 460 nm. 
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In still other illustrative embodiments, the optical signal is a video signal, Raman signal, or 
infrared signal. Some illustrative embodiments include using difference spectra calculated 
between different phases of acetowhitening, using various normalization schema, and/or using 
various combinations of spectral data and/or image data as discussed above. 
[0398] In one preferred embodiment, determining an optimal window for obtaining spectral 
data in step 104 of Figure 1 includes developing linear discriminant analysis models using 
spectra from each time bin shown in Table 1 below. 

Table 1; Time bins for which mea ns spectra arc obtained in an exemplary embodimgnt 



Bin 


Time after aDDlication of Acetic Acid Csl 


1 


t<0 


2 


0<t<40 


3 


40<t<60 


4 


60<t<80 


5 


80<t<100 


6 


, 100<t<120 


7 


120<t<140 


8 


140<t<160 


9 


160<t<180 


10 


t>180 



[0399] Alternatively, nonlinear discriminant analysis models may be developed. Generally, 
models for the determination of an optimal window are trained using reflectance and 
fluorescence data separately, although some embodiments include using both data types to train 
a model. The discriminant analysis models discussed herein for exemplary embodiments of the 
detemiination of an optimal window are generally less sophisticated than the schema used in the 
tissue classification module 132 in Figure 1. Alternatively, a model based on the tissue 
classification schema m the module 132 in Figure 1 can be used to detennine an optunal window 
for obtainmg spectral data in step 1 04 of Figure 1 . 

[0400] In exemplary embodunents for determming an optimal window discussed herein, 
reflectance and fluorescence intensities are down-sampled to one value every 10 nm between 
360 and 720 nm. A model is trained by adding and removing intensities in a forward manner 
continuously repeating the process until the model converges such that additional intensities do 
not appreciably unprove tissue classification* Testing is performed by a leave-one-spectrum-out 
jack-knife process. 
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[0401] Figure 35 shows the difference between the mean reflectance spectra for non-CIN 2/3 
tissues and CIN 2/3 tissues at three times (prior to the application of acetic acid (graph 976), 
maximum whitenmg (graph 978, about 60 - 80 seconds post-AA), and the last time data were 
obtamed (graph 980, about 160 - 180 seconds post-AA)). The time corresponding to maximum 

5 whitening was determined from reflectance data, and occurs between about 60 seconds and 80 
seconds following application of acetic acid. In the absence of acetic acid, the reflectance 
spectra for CIN 2/3 (curve 982 of graph 976 in Figure 35) are on average lower than non-CIN 
2/3 tissue (curve 984 of graph 976 in Figure 35). Following the application of acetic acid, a 
reversal is noted - CIN 2/3 tissues have higher reflectance than the non-CIN 2/3 tissues. The 

10 reflectance of CIN 2/3 and non-CIN 2/3 tissues mcrease with acetic acid, with CIN 2/3 showing 
a larger relative percent change (compare curves 986 and 988 of graph 978 m Figure 35). From 
about 160 s to about 180 s foUowmg acetic acid, the reflectance of CIN 2/3 tissue begins to 
return to the pre-acetic acid state, while the reflectance of the non-CIN 2/3 group continues to 
increase (compare curves 990 and 992 of graph 980 in Figure 35). 

15 [0402] Discrimination function 'spectm' are calculated from the reflectance spectra of CIN 2/3 
and non-CIN 2/3 tissues shown in Figure 35 as one way to determine an optimal window for 
obtaining spectral data. Discrimination function spectra comprise values of the discrimination- 
function in Equation 76 determined as a function of wavelength for sets of spectral data obtained 
at various times. As shown in Figure 36, the largest differences (measured45y the largest 

20 absolute values of discrimmation function) are found about 60 s to about 80 s post-acetic acid 
(curve 1002), and these data agree with the differences seen in the mean reflectance spectra of 
Figure 35 (curves 986 and 988 of graph 978 in Figure 35). 

[0403] Multivariate linear regression analysis takes into account wavelength mtodependencies 
. in determming an optimal data acquisition window. One way to do this is to classify spectral 

25 data shown in Figure 35 using a model developed from the reflectance data for each of the bins 
in Table 1 . Then, the accuracy of the models for each bin is computed and compared. 
Reflectance intensities are down-sampled to one about every 1 0 nm between about 360 nm and 
about 720 nuL The model is trained by adding intensities in a forward-stepped manner. Testing 
is performed with a leave-one-spectrum-out jack-knife process. The results of the linear 

30 regression show which wavelengths best separate CIN 2/3 from non-CIN 2/3, as shown in Table 
2. 
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Tabie2; Forwarded selected best reflectance wavelengths for classifying CIN 2/3 
from non-CIN 2/3 spectra obtained at different times pre and post-AA. 



Time from AA (s) 


LDA Model Input Wavelengths (nm) 


Accuracy 


-30 


370 400 420 440 530 570 590 610 


66 


30 


420430 450 600 


74 


50 


360 400420 430 580 600 


74 








90 


360 420 430 540 590 


73 


110 


360440 530 540 590 


71 


130 


360 420 430 540 590 


71 


150 


370 400 430 440 540 620 660 690 720 


72 


' ""Afo-^ 


: , 490 530 570 630 650 " • 





[0404] As shovra in Table 2, the two best models for separating CIN 2/3 and non-CIN 2/3, 
taking into account wavelength interdependence, use reflectance data obtained at peak CIN 2/3 
5 whitening (Gmm about 60s to about 80s) and reflectance data obtained from about 160s to about 
1 80s post acetic acid. The first model uses input wavelengths between about 360 and about 600 
lun, while the second model uses more red-shifted wavelengths between about 490 and about 
650 nm. This analysis shows that the optimal windows are about 60s-80s post AA and about 
160-1 80 post AA (the latest time bin). This is consistent with the behavior of the discrimination 

10 function spectra shown in Figure 6. 

[0405] Figure 37 demonstrates one step in determining an optimal window for obtaining 
spectral data, for purposes of discriminating between CIN 2/3 and non-CIN 2/3 tissue. Figure 37 
shows a gr^h 1 006 depicting the performance of the two LDA models described in Table 2 
above as applied to reflectance spectral data obtained at various times following application of 

15 acetic acid 1008. Curve 1010 in Figure 37 is aplot of the diagnostic accuracy of the LDA model 
based on reflectance spectral data obtained between about 60 and about 80 seconds ("peak 
whitening model") as applied to reflectance spectra from the bins of Table 1, and curve 1012 in 
Figure 37 is a plot of the diagnostic accuracy of the LDA model based on reflectance spectral 
data obtained between about 160 and about 1 80 seconds, as applied to reflectance spectra from 

20 the bms of Table 1 . For the peak-whitening model, tiie highest accuracy was obtained at about 
70 s, while accuracies greater than 70% were obtained with spectra collected in a window 
between about 30s and about 130s. The 160-180 s model had a narrower window around 70 s, 
but performs better at longer times. 

[0406] Figure 38 shows tiie difference between the mean 337-nm fluorescence spectra for non- 
25 CIN 2/3 tissues and CIN 2/3 tissues at three times ^rior to application of acetic acid (gr^h 
1014), maximum whitening (graph 1016, about 60 to about 80 seconds post-AA), and at a time 
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corresponding to the latest time period in which data was obtained (graph 1018, about 160 to 
about 180 seconds post-AA)). The time corresponding to maximum whitening was determined 
from reflectance data, and occurs between about 60 seconds and 80 seconds following 
application of acetic acid. In the absence of acetic acid, the fluorescence spectra for CIN 2/3 

5 tissue (curve 1020 of graph 1014 in Figure 38) and for non-CIN 2/3 tissue (curve 1022 of graph 
1014 in Figure 38) are essentially equivalent wifli a slightiy lower fluorescence noted around 390 
nm for CIN 2/3 sites. Following tiie application of acetic acid, tiie fluorescence of CIN 2/3 and 
non-CIN 2/3 tissues deorease, with CIN 2/3 showing a larger relative percent change (compare 
curves 1024 and 1026 of graph 1016 in Figure 38), From about 160s to about 180 s following 

10 acetic acid application, the fluorescence of CIN 2/3 tissue shows signs of returning to the pre- 
acetic acid state while the fluorescence of the non-CIN 2/3 group continues to decrease (compare 
curves 1028 and 1030 of graph 1018 in Figure 38). 

[0407J An optimal data acquisition window may also be obtained using a discrimination 
function calculated from fluorescence spectrei of CIN 2/3 and non-CIN 2/3 tissues shown in 

15 Figure 38. In one example, discrimination function spectra include values of tiie discrimination 
function in Eqiiation 76 determined as a function of wavelengtti for sets of spectral data obtained 
at various times. Figure 39 shows a graph 1032 depicting the discrimination function spectra 
evaluated usmg the fluorescence data of Figure 38 obtained prior to application of acetic acid, 
and at two times post-AA. As shown in Figure 39, applications of acetic acid improves that 

20 distmction between CIN 2/3 and non-CIN 2/3 tissues using fluorescence data. The largest 
absolute values are found using data measured within the range of about 160-180 s post-acetic 
acid (curve 1042), and these agree with tiie differences sem in the mean fluorescence spectra of 
Figure 38 (curves 1030 and 1028 of graph 1018 m Figure 38). 

[0408] Multivariate Imear regression takes into account wavelengtii interdependencies in 
25 determining an optimal data acquisition window. An application of one method of determining 
an optimal window includes classifying data represented in the CIN 2/3, CIN 1, and NED 
categories m the Appendix Table into CIN 2/3 and non-CIN 2/3 categories by using 
classification models developed firom the fluorescence data shown in Figure 38. Fluorescence 
mtensities are down-sampled to one about every 10 nm between about 360 and about 720 nm. 
30 The model is trained by adding intensities in a forward manner. Testing is performed by a leave- 
one-spectrum-out jack-knife process. The result of this analysis shows which wavelengths best 
separate CIN 2/3 from non-CIN 2/3, as shown in Table 3. 
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ssibles: Forwarded selected best 337-nin fluorescence wavelengths for 
classifying CIN 2/3 from non-CIN 2/3 spectra obtained at different times pre 
and post-AA. 



Time from /\A(s) 


LDA Mode! Input Wavelengths (nm) 


Accuracy 


-30 


380, 430, 440, 610. 660, 700, 710 


61 


30 


370, 380. 390. 640 


61 


50 


410 


54 




mmmmmmmmmmmmm 




90 


370, 380, 420, 460, 500. 560, 660 


64 


110 


360, 390, 400, 710 


51 


130 


370 


53 


150 


370, 380, 440, 620, 640, 700 


65 


: ■ .:• T70 r " ■ 


370. 480, 51b. 570. 600. 700. 720 





5 [0409] As shown in Table 3, the two best models for separatmg CIN 2/3 and non-CIN 2/3, 
taking into account wavelength interdependencies, use data obtamed at peak CIN 2/3 whitening 
(60-80 s) and data obtained at ihe latest time measured (from about 160s to about 1 80 s post 
acetic acid). The first model uses input wavelengths between about 360 and about 670 nm, 
while the second model uses wavelengths between about 370 and about 720 mn. 

10 [0410] Figure 40 demonstrates one step in determining an optimal window. Figure 40 shows a 
graph 1044 depicting the performance of the two LDA models described in Table 3 above as 
applied to fluorescence spectral data obtained at various times following application of acetic 
acid 1046. Curve 1048 in Figure 40 is a plot of the diagnostic accuracy of the LDA model based 
on fluorescence spectral data obtained between about 60 and about 80 seconds C*peak whitening 

15 model") as applied to fluorescence spectra from the bins of Table 1, and curve 1050 in Figure 40 
is a plot of the diagnostic accuracy of the LDA model based on fluorescence spectral data 
obtained between about 160 and about 180 seconds, as applied to fluorescmce spectra from the 
bins of Table 1 . The accuracies of these models vary depending on when the fluorescence 
spectra are recorded relative to the application of acetic acid, as shown in Figure 40. The 

20 predictive ability of the fluorescence models in Figure 40 tend to be less than that of the 
reflectance models in Figure 37. Accuracies greater than 70% are obtained with spectm 
collected after about 1 60 seconds post-AA. 

[0411] One embodunent includes classifying spectral data shown in Figure 38 from known 
reference tissue samples into CIN 2/3 and non-CIN 2/3 categories by using classification models 
25 developed from the fluorescence data for each of the bins in Table 1 . Models are developed 
based on time post acetic acid. Ratios of fluorescence to reflectance are down-sampled to one 
every 10 nm between about 360 and about 720 nm. The model is tramed by adding intensities in 
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a forward manner. Testing is perfornoied by a leave-one-spectnm-out jack-knife pro^^^ For 
this analysis, the model is based on intensities at about 360, 400, 420, 430, 560, 610, and 630 
nm. In general, the results are slightly better than a model based on fluorescence alone. 
Improved performance is noted from spectra acquired at about 160 s post acetic acid. 
5 [0412] Figure 41 shows a graph 1052 depicting the accuracy of three LDA models as applied 
to spectral data obtained at various times following application of acetic acid 1054, used in 
determining an optimal window for obtaining spectral data. Curve 1056 in Figure 41 is a plot of 
the diagnostic accuracy of the LDA model based on reflectance spectral data obtained between 
about 60 and about 80 seconds ("peak whitening model"), also shown as curve 1010 in Figure 

10 37. Curve 1058 in Figure 41 is a plot of the diagnostic accuracy of the LDA model based on 
fluorescence spectral data obtained between about 60 and about 80 seconds C'peak \Adutening 
model"), also shown as curve 1048 in Figure 40. Curve 1060 in Figure 41 isaplotofthe 
diagnostic accuracy of the LDA model based on fluorescence intensity divided by reflectance, as 
described in flie iromediately preceding paragraph. 

15 [0413] The exemplary embodiments discussed above and illustrated in Figures 35 to 41 

provide a basis for selecting an optimmn window for obtaining spectral data upon application of 
acetic acid. Other factors to be considered include the time required to apply the contrast agent 
and to perform target focusing as shown in Figure 27 A. Another factor is the time required to 
perform a scan over a sufficient number of regions of a tissue sample to provide an adequate 

20 indication of disease state with sufficient sensitivity and selectivity. Also, a consideration may 
be made for the likelihood of tiie need for and time required for retakes due to patient motion. 
[0414] The factors and analysis discussed above indicate that an optimal data acquisition 
window is a period of time from about 30 seconds following application of a contrast agent (for 
example, a 5 volume percent acetic acid solution) to about 130 seconds following application of 

25 the contrast agent. Other optimal windows are possible. For example, one alternative 

embodiment uses an optimal window with a "start" time from about 10 to about 60 seconds 
following application of acetic acid, and an "end" time from about 1 10 to about 180 seconds 
foUowdng application of acetic acid. 

[0415] An alternative maimer for determining an optimal window comprises determining and 
30 using a relative amplitude change and/or rate of amplitude change as a trigger for obtaining 

spectral data from a sample. By using statistical and/or heuristic methods such as those discussed 
herein, it is possible to relate more easily-monitored relative changes or rates-of-change of one or 
more optical signals from a tissue sample to corresponding fiill spectrum signals that can be used 
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in characterizing the state of health of a given sample. For example, by performing a 
discrimination function analysis, it may be found for a given tissue type that when the relative 
change in reflectance at a particular wavelength exceeds a threshold value, the corresponding 
full-spectrum reflectance can be obtained and then used to accurately classify the state of health 
5 of the tissue. In addition, the tri^ers determined above may be converted into optimal time 
windows for obtaining diagnostic optical data from a sample. 

[0416] Figure 42 shows how an optical amplitude trigger is used to determine an optimal time 
window for obtaining diagnostic optical data. The graph 1062 m Figure 42 plots the normalized 
relative change of mean reflectance signal 1064 from tissue samples with a given state of health 

10 as a function of time following application of acetic acid 1066. The mean reflectance signal 
determined from CIN 1, CIN 2, and Metaplasia samples are depicted in Figure 42 by curves 
1068, 1070, and 1072, respectively. Figure 42 shows that when the normalized relative change 
of mean reflectance reaches or exceeds 0.75 in this example, die image intensity data and/or the 
full reflectance and/or fluorescence spectrum is most indicative of a given state of health of a 

15 sample. Thus, for CIN 2 samples, for example, this corresponds to a tune period between t] and 
t2, as shown in the graph 1062 of Figure 42. Therefore, spectral and/or image data obtained from 
a tissue sample between ti and ta following application of acetic acid are used in accurately 
determining whether or not CIN 2 is indicated for that sample. In one embodiment, the relative 
change of reflectance of a tissue sample at one or more given wavelengths is monitored. When 

20 that relative change is greater than or equal to the 0.75 threshold, for example, more 

comprehensive spectral and/or image data are obtamed to characterize whether the sample is 
indicative of CIN 2. In another embodiment, a predetermined range of values of the relative 
optical signal change is used such that when the relative signal change falls within the 
predetermined range of values, additional spectral and/or image data is captured in order to 

25 characterize the sample. 

[0417] Figure 43 shows how a rate-of-change of an optical amplitude trigger is used to 
determine an optimal time window for obtaining diagnostic optical data. The graph 1074 of 
Figure 43 plots the slope of an exemplary mean reflectance signal 1076 from tissue samples with 
a given state of health as a function of time following application of acetic acid 1078. The slope 

30 of mean reflectance is a measure of the rate of change of the mean reflectance signal. The rate of 
change of mean reflectance determined from CIN 1, CIN 2, and metaplasia samples are depicted 
in Figure 43 by curves 1080, 1082, and 1084, respectively. Those curves show that when the 
absolute value of the slope is less tiian or equal to 0.1, for example, in the vicinity of maximum 
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reflectance, the image intensity data and/or the full reflectance and/or fluorescence spectrum is 
most indicative of a given state of health of a sample. Thus, for CIN 2 samples, for example, this 
corresponds to a time period between ti and tj as shown in tiie graph 1074 of Figure 43. 
Therefore, spectral and/or image data obtained from a tissue sample between ti and t2 following 
5 application of acetic acid is used in accurately determining vdiether or not CIN 2 is indicated for 
that sample. In the example, the rate of change of reflectance of a tissue sample is monitored at 
one or more wavelengths. When that rate of change has an absolute value less than or equal to 
0.1, more comprehensive spectral and/or unage data are obtained from the sample for purposes 
of characterizing whether or not the sample is indicative of CIN 2. Figure 43 demonstrates use 
10 of a range of values of rate of optical signal change. Other embodiments use a single threshold 
value. 

Motion tracking 

I0418J In one embodiment, the tissue characterization system shown in Figure 1 comprises 
real-time motion tracking (step 106 in Figure 1). Real-time tracking determines a correction for 

15 and/or compensates for a misalignment between two images of the tissue sample obtained during 
a spectral data scan (i.e. step 732 in Figure 27A and 27B), where the misalignment is caused by a 
shift in the position of the sample with respect to the instrument 102 in Figure 1 (or, more 
particularly, the probe optics 178). The misalignment may be caused by unavoidable patient 
motion, such as motion due to breathing during the spectral data scan 732. 

20 [0419] In one embodiment, the con-ection factor determined by the real-time tracker is used to 
automatically conq)ensate for patient motion, for example, by adjusting the instrument 102 
(Figure 1) so thiat spectral data obtained from mdexed regions of tiie tissue sample during the 
scan correspond to their originally-indexed locations. Alternatively or additionally, the motion 
correction fector can be used in q)ecti:al data pre-processing, stqp 1 14 in Figure 1 and Figure 11, 

25 to correct spectral data obtamed during a scan according to an applicable correction factor. For 
example, tiie spectral data lookup method in step 1 14 of Figure 1 as discussed herein may 
compensate for patient motion by using a correction determined by tiie real-time tracker 1 06 to 
correlate a set of spectral data obtained during a scan with its true, motion-corrected position 
(x,y) on the tissue sample. In one embodiment, the motion correction factor deteamined in step 

30 106 of Figure 1 is updated about once every second during the scan using successive images of 
the tissue, as shown m Figure 27B. Step 106 determines and validates a motion con«ction factor 
about once every second during the spectral scan, corresponding to each successive image in 
Figure 27B. Then, the pre-processing component 1 14 of Figure 1 corrects the spectral data 
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obtained at an interrogation point during the spectral scan using the correction factor 
corresponding to the time at which the spectral data were obtained. 

[0420] A typical misalignment between two images obtained about 1 second apart is less than 
about 0.55-mm within a two-dimensional, 480 x 500 pixel image frame field covering a tissue 

5 area of approximately 25-mm x 25-mm. These dimensions provide an example of the relative 
scale of misaligmnent versus image size« In some instances it is only necessary to compensate 
for misalignments of less than about one millimeter within the exemplary image frame field 
defined above. In other cases, it is necessary to compensate for misalignments of less than about 
0.3-mm within the exemplary image frame field above. Also, the dimensions represented by the 

10 image frame field, the nimiber of pixels of the image frame field, and/or the pixel resolution may 
differ from the values shown above. 

[0421] A misalignment correction determination may be inaccurate, for example, due to any 
one or a combination of the following: non-translational sample motion such as rotational 
motion, local deformation, and/or warpmg; changing features of a sample such as whitening of 

15 tissue; and image recording problems such as focus adjustmmt, missing images, blun^d or 

distorted images, low signal-to-noise ratio, and computational artifacts. Validation procedures of 
the invention identify such inaccuracies. The methods of validation may be conducted "on-the- 
fly" in concert with the methods of determining misalignment corrections in order to improve 
accuracy and to reduce the time required to conduct a given test. 

20 [0422] In order to facilitate the automatic analysis m the tissue classification system 100 of 
Figure 1, it is often necessary to adjust for misalignments caused by tissue sample movement 
that occurs during the diagnostic procedure. For example, during a given procedure, in vivo 
tissue may spatially shift within the image firame field from one ims^e to the next due to 
movement of the patient Accurate tissue characterization requkes that this movement be taken 

25 into account in the automated analysis of the tissue sample. In one embodiment, spatial shift 
correction made throughout a spectral data scan is more accurate flian a correction made after the 
scan is complete, since "on-the-fly" corrections compensate for smaller shifts occurring over 
shorter periods of time and since spectral data is being continuously obtained throughout the 
approximately 12 to 15 second scan in the embodiment of Figure 27B. 

30 [0423] If a sample moves while a sequence of images is obtained, the procedure may have to 
be repeated For example, this may be because the shift between consecutive images is too large 
to be accurately compensated for, or because a region of interest moves outside of a usable 
portion of the firame c^tured by the optical signal detection device. Stepwise motion correction 
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of Spectral data reduces the cumulative effect of sample movement If correction is made only 
after an entire sequence is obtained, it may not be possible to accurately compensate for some 
types of sample movement. On-the-fly, stepwise compensation for misalignment reduces the 
need for retakes. 

5 [04241 On-the-fly compensation may also obviate the need to obtain an entire sequence of 
images before makmg the decision to abort a Med procedure, particularly when coupled wiih 
on-the-fly, stepwise validation of the misalignment correction detemunation. For example, if flie 
validation procedure detects that a misalignment correction determmation is either too large for 
adequate compensation to be made or is invalid, the procedure may be aborted before obtaining 

10 the entire sequence of images. It can be immediately determined whether or not the obtained 
data is useable. Retakes may be performed during the same patient visit; no follow-up visit to 
repeat an erroneous test is required. A diagnostic test invalidated by excessive movement of the 
patient may be aborted before obtaining the entire sequence of images, and a new scan may be 
completed, as long as there is enough remaining time m the optimal time window for obtaining 

15 spectral data. 

[0425] In preferred embodiments, a determination of misalignment correction is expressed as a 
translational displacement in two dimensions, x and y. Here, x and y represent Cartesian 
coordinates mdicating displacement on tiie image jframe field plane. In other embodiments, 
corrections for misalignment are expressed in terms of non-Cartesian coordinate systems, such as 

20 biradical, spherical, and cylindrical coordinate systems, among others. Alternatives to Cartesian- 
coordinate systems may be useful, for example, where the image frame field is non-planar, 
[0426] Some types of sample motion - including rotational motion, warpmg, and local 
deformation - may result in an invalid misalignment correction determination, since it may be 
impossible to ejqpress certain mstances of these types of sample motion m terms of a translational 

25 displacement, for example, in the two Cartesian coordinates x and y. It is noted, however, that in 
some embodiments, rotational motion, warping, local deformation, and/or other kinds of non- 
translational motion are acceptably accounted for by a correction expressed in terms of a 
translational displacement. The changing features of the tissue, as m acetowhitening, may also 
affect the determination of a misalignment correction. Image recording problems such as focus 

30 adjustment, missing images, blurred or distorted images, low signal-to-noise ratio (i.e. caused by 
glare), and computational artifacts may affect the correction determination as well. Therefore, 
validation of a determined correction is often required. In some embodiments, a validation step 
includes determining whether an individual correction for misaligrmient is erroneous, as well as 



wo 2004/095359 



PCT/US2004/011820 



-96- 

determining whether to abort or continue the test in progress. Generally, validation comprises 
splitting at least a portion of each of a pair of images into smaller, corresponding units 
(subimages), determining for each of these smaller units a measure of the displacement that 
occurs within the unit between the two images, and comparing the unit displacements to the 

S overall displacement between the two images. 

[0427] In certain embodunents, the metiiod of validation takes into accoimt the fact that 
features of a tissue sample may change during the capture of a sequence of images. For 
example, the optical intensity of certain regions of tissue change during the approximately 12 to 
15 seconds of a scan, due to acetowhitening of the tissue. Therefore, in one embodiment, 

10 validation of a misalignment correction determination is performed using a pair of consecutive 
images. In this way, the difference between the corresponding validation cells of the two 
consecutive images is less affected by gradual tissue whitening changes, as compared with 
images obtained further apart in time. In an alternative embodiment, validation is performed 
using pairs of nonconsecutive images taken within a relatively short period of time, compared 

IS with the time in which the overall sequence of images is obtained. In other embodiments, 
validation comprises the use of any two images in the sequence of imj^es. 
[0428] A determination of misalignment correction between two images is inadequate if 
significant portions of the images are featureless or have low signal-to-noise ratio (i.e. are 
affected by glare). Similarly, validation using cells containing significant portions that are 

20 featureless or that have low signal-to-noise ratio may result in the erroneous invalidation of valid 
misalignment correction determinations. This may occur in cases where the featureless portion 
of the overall image is small enough so that it does not adversely affect the misalignment 
correction determination. For example, analysis of featureless validation cells may produce 
meaningless correlation coefficients. One embodiment includes identifying one or more 

25 featureless cells and eliminating tiiem from consideration in the validation of a misalignment 
correction determination, thereby preventing rejection of a good misalignment correction. 
[0429] A determination of misalignment correction may be erroneous due to a computational 
artifact of data filtering at the image borders. For example, in one exemplary embodiment, an 
image with large intensity differences between the upper and lower borders and/or the left and 

30 right borders of the image frame field undergoes Laplacian of Oaussian frequency domain 
filtering. Since Laplacian of Gaussian frequency domain filtering corresponds to cyclic 
convolution in tiie space-time domain, these intensity differences (discontinuities) yield a large 
gradient value at the image border, and cause tiie overall misalignment correction determination 
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to be erroneous, since changes between the two images due to spatial shift are dwarfed by the 
edge effects. One alternative embodiment employs pre-multiplication of image data by a 
Hamming window to remove or reduce this 'Svraparound error." However, one preferred 
embodiment employs an image-blending technique such as feathering, to smooth any border 

5 discontinuity, while requuing only a minimal amount of additional processing time. 

[0430] Figure 44A rq)resents a 480 x 500 pixel image 1086 &om a sequence of images of in 
vivo human cervix tissue and shows a 256 x 256 pixel portion 1088 of the image that the motion 
correction step 106 in Figure 1 uses in identifying a misalignment correction between two 
images from a sequence of images of the tissue, according to one embodiment The image 1086 

10 of Figure 44A has a pixel resolution of about 0.054-mm. The embodiments described herein 
show images with pixel resolutions of about 0.0547-mm to about 0.0537-mm. Other 
embodiments have pixel resolutions outside this range. In some embodiments, the images of a 
sequence have an average pixel resolution of between about 0.044-mm and about 0.064-mm. In 
the embodiment of Figure 44A, step 106 of the system of Figure 1 uses the central 256 x 256 

15 pbcels 1088 of the image 1086 for motion tmcking. An alternative embodiment uses a region of 
different size for motion tracking, which may or may not be located in the center of the image 
frame field. In the embodiment of Figure 44A, the motion tracking stsp 106 of Figure 1 
determines an x-displacement and a y-displacement corresponding to the translational shift 
(misalignment) between the 256 x 256 central portions 1088 of two images in the sequence of 

20 images obtained during a patient spectral scan. 

[0431] The determination of misalignment correction may be erroneous for any number of 
various reasons, including but not limited to non-translational sample motion (i.e. rotational 
motion, local deformation, and/or warping), changing features of a sample (i.e. whitening of 
tissue), and image recording problems such as focus adjustment, missing images, blurred or 

25 distorted images, low signal-to-noise ratio, and computational artifacts. Therefore, in preferred 
embodiments, validation comprises splitting an image into smaller units (called cells), 
determining displacements of these cells, and comparing the cell displacements to the overall 
displacement. Figure 44B depicts the image represented in Figure 44A and shows a 128 x 128 
pixel portion 1090 of the image, made up of 16 individual 32 x 32 pixel validation cells 1092, 

30 from which data is used to validate the misaligmnent correction, 

[0432] Figure 45, Figures 46A and B, and Figures 47A and B depict steps in illustrative 
embodiment methods of determining a misaligmnent correction between two images of a 
sequence, and methods of validating tiiat determination. Steps 1096 and 1098 of Figure 45 show 
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development of data from an initial image with which data from a subsequent image are 



compared in order to determine a misaligmnent correction between the subsequent image and the 
initial image. An initial image "o" is preprocessed, then filtered to obtam a matrix of values, for 
example, optical luminance (brightness, intensity), representing a portion of the initial image. In 

S one embodiment, preprocessing comprises transforming the three RGB color components 

corresponding to a given pixel into a single luminance value. An exemplary luminance is CCIR 
601, shown in Equation 63. CCIR 601 luminance may be used, for example, as a measure of the 
'Svhiteness" of a particular pixel in an image from an acetpwhitening test Different expressions 
for grayscale luminance may be used, and the choice may be geared to the specific type of 

10 diagnostic test conducted. The details of step 1 096 of Figure 45 is illustrated m blocks 1130, 
1 132, and 1 134 of Figure 46A, where block 1 130 represents the mitial color hnage, "o", in the 
sequence, block 1 1 32 represents conversion of color data to grayscale using Equation 63, and 
block 1 134 represents the image of block 240 after conversion to grayscale. Referring now to 
Figures 46A and 46B, Figure 46B is a continuation of Figure 46A, linked, for example, by the 

15 curded coimectors labeled A and B. Accordmgly, going forward. Figures 46A and 46B are 
referred to as Figure 46. 

[0433] Step 1098 of Figure 45 represents filtering a 256 x 256 portion of the initial image, for 
example, a portion analogous to the 256 x 256 central portion 1088 of the image 1086 of Figure 
44 A, using Laplacian of Gaussian filtering. Other filtering techniques are used in other 

20 embodiments. Preferred embodiments employ Laplacian of Gaussian filtering, which combines 
the Laplacian second derivative approximation with the Gaussian smoothing filter to reduce the 
high frequency noise components prior to differentiation. This filtering step may be performed 
by discrete convolution in the space domain, or by frequency domain filtering. The Laplacian of 
Gaussian (LoG) filter may be expressed in terms of x and y coordmates (centered on zero) as 

25 shown in Equation 77: 



where x and y are space coordinates and a is the Gaussian standard deviation. In one preferred 
embodiment, an approximation to the LoG fimction is used. Illustrative embodiments described 
herehi include use of an approxuriation kemel(s) of size 9 x 9, 21 x 21 , and/or 31x31. The 
30 Gaussian standard deviation, a, is chosen in certain preferred embodiments iising Equation 78: 




o = LoG filter size 78.49 



(78) 
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where LoG filter size corresponds to the size of the discrete kernel approximation to the LoG 
function (Le. 9, 21, and 3 1 for the approximation kernels used herein). Other embodiments 
employ difierent kernel approximations and/or different values of Gaussian standard deviation. 
[0434] The LoG filter size may be chosen so that invalid scans are foiled and valid scans are 

5 passed with a minimum of error. Generally, use of a larger filter size is better at reducing large 
structured noise and is morei sensitive to larger image features and larger motion, while use of a 
smaller filter size is more sensitive to smaller features and smaller motion. One embodiment of 
the invention comprises adjusting filter size to coordmate with the kind of motion being tracked 
and the features being imaged, 

10 [04351 The details of step 1098 of Figure 45 is illustrated in Figure 46 in blocks 1134, 1136, 
and 1138 where block 1 134 represents data from the initial image in the sequence after 
conversion to grayscale luminance, block 1 136 represents the application of the LoG filter, and 
block 1138 represents the 256 x 256 matrix of data values, Go(x,y), which is the "gold standard" 
by which other uuages are compared in validating misalignment correction detemiinations in this 

15 , embodiment As detailed in Figures 47A and 47B, one embodiment validates a misalignment 
correction determination by comparing a given image to its preceding image in the sequence, not 
by comparing a given image to the initial image in the sequence as shown in Figure 46. 
(Referring now to Figures 47A and 47B, Figure 47B is a continuation of Figure 47A, linked, for 
example, by the circled connectors labeled A, B, and C. Accordingly, going forward. Figures 

20 47A and 47B are referred to as Figure 47.) Although Figure 45, Figure 46, and Figure 47 show 
application of the LoG filter as a discrete convolution in the space domain, resulting in a 
standard expressed in space coordinates, other embodiments comprise £^plying tiie LoG filter in 
the frequency domain. In either case, the LoG filter is preferably zero padded to the image size. 
[0436] The details of steps 1100 and 1 102 of Figure 45 represent preprocessing an image "i" 

25 by converting RGB values to grayscale luminance as discussed above, and performing LoG 
filtering to obtain Gi(x,y), a matrix of values from image which is compared with that of 
another image in the sequence in order to determine a misalignment correction between the two 
images. The details of steps 1 100 and 1 102 of Figure 45 are illxistrated in Figure 46 in blocks 
1 140, 1 142, 1 144, 1 146, and 1 148, where fi(x,y) in block 1 140 is the raw image data from image 

30 "i", block 1 142 represents conversion of the fi(x,y) data to gray scale intensities as shown in 
block 1 144, and block 1 146 represents application of tiie LoG filter on the data of block 1 144 to 
produce the data of block 1 148, Gi(x,y). 
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[0437] Similarly, steps 1 106 and 1 108 of Figure 45 represent preprocessing an image "j" by 
converting RGB values to grayscale luminance as discussed above, and performing LoG filtering 
to obtain Gj(x,y), a matrix of values from image "j" which is compared with image "i" in order to 
determine a measnre of misalignment between the two images. In some preferred embodiments, 
5 image "j" is subsequent to image *Y' in the sequence. In some preferred embodiments, "i" and 
"j" are consecutive unages. Steps 1 106 and 1 108 of Figure 45 are illustrated in Figure 46 in 
blocks 1 154, 1 156, 1 158, 1 160, and 1 162, where "j" is *H+r', the image consecutive to image "i" 
in the sequence. In Figure 46, block 1 154 is the raw "i+1" image data, block 1 156 represents 
conversion of the "i+l" data to gray scale intensities as shown in block 1 158, and block 1 160 
10 represents application of the LoG filter on the data of block 1 158 to produce the data of block 
1162,Gw(x,y). 

[0438] Steps 1 1 04 and 1 1 1 0 of Figure 45 represent applying a Fourier transform, for example, 
a Fast Fourier Transform (FFT), usmg Gi(x,y) and Grj(x,y), respectively, to obtain Fi(u,v) and 

which are maMces of values in tiie frequency domain corresponding to data from images 

15 "i" and "j^ respectively. Details of steps 1 104 and 1 1 10 of Figure 45 are illustrated in Figure 46 
by blocks 1 148, 1 150, 1 152, 1 162, 1 164, and 1 166, where "j" is "i+l", the image consecutive to 
image "i" in the sequence. In Figure 46, block 1 148 represents the LoG filtered data, Gi(x,y), 
correspondmg to image "i", and block 1150 represents taking the Fast Fourier Transform of 
Gi(x,y) to obtain Fi(u,v), shown in block 1 152. Similarly, in Figure 46 block 1 162 is the LoG 

20 filtered data, Gi+i(x,y), corresponding to image "i+1 and block 1 1 64 represents taking the Fast 
Fourier Transform of GH-i(x,y) to obtain Fi+i(u,v), shown in block 1 166. 
[0439] Step 1112 of Figure 45 represents computing the cross correlation F|(u,v) F*j(u,v), 
where Fi(u,v) is the Fourier transform of data from image "i*', F*j(u,v) is the complex conjugate 
of the Fourier transform of data from image "j*', and u and v are frequency domain variables. 

25 The cross-correlation of two signals of lengthNi andN2 provides N1+N2-I values; thus avoiding 
aliasing problems due to under-sampling, the two signals should be padded with zeros up to 
N1+N2-I samples. Details of step 1 1 12 of Figure 45 are rq)resented m Figure 46 by blocks 
1 152, 1 166, and 1 168. Block 1 168 of Figure 46 represents computing the cross correlation, 
Fi(u,v) F*i+i(u,v), using Fi(u,v), the Fourier transform of data from image "i", and F*i+i(u,v), the 

30 complex conjugate of the Fourier transform of data from image "i+l". The cross-correlation 
may also be expressed as c(k,I) in Equation 79: 

c(kl) = S 2 Ii(p»q)h(P'k,q'l) (79) 
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where variables (k,l) can be thought of as the shifts in each of the x- and y-directions which are 
being tested in a variety of combinations to determine the best measure of misaligmnent between 
two images Ii and /i, and where p and q are matrix element markers. 

[0440] Step 1114 of Figure 45 represents computing the mverse Fourier transform of flie cross- 
5 correlation computed in step 1 1 12. Step 1 1 14 of Figure 45 is represented m Figure 46 by block 
1 170. The resulting mverse Fourier transform maps how well tiie 256 x 256 portions of images 
"i" and "j" match up with each other given various combinations of x- and y-shifts. Generally, 
the normalized correlation coeflBcient closest to 1.0 corresponds to the x-shift and y-shift 
position providing the best match, and is determined from the resulting inverse Fourier 

10 transform. In a preferred embodiment, correlation coefBcients are normalized by dividing 
matrix values by a scalar computed as tiie product of the square root of the (0,0) value of the 
awto-correlation of each image. In this way, variations in overall brightness between the two 
unages have a more limited effect on the correlation coefficient, so that the actual movement 
within the image frame field between the two unages is better reflected in the misaligmnent 

IS determinatioiL ' 

10441] Step 1 1 16 of Figure 45 represents determining misalignment values dx, dy, d, sum(djO, 
sum(dy), and Sum(dj), where dx is the computed displacement between the two images "i" and 
"j" in the x-direction, dy is the computed displacement between the two images in the y- 
direction, d is the square root of the sum dx^+dy^ and represents an overall displacement between 

20 the two images, sum(dx) is the cumulative x-displacement between the current image "j" and tiie 
first image in the sequence "o", sum(dy) is the cumulative y-displacement between the current 
image "j" and the first hnage in the sequence "o", and Sum(dj) is the cumulative displacement, d, 
between the cunrrat image "j" and the first inu^e m the sequence "o". Step 1 1 16 of Figure 45 is 
represented m Figure 46 by blocks 1 172, 1 174, and 1 176. Blocks 1 174 and 1 176 represent 

25 finding the maxunum value m the data of block 1 172 in order to calculate dx, dy, d, sum(dx), 
sum(dy), and Sum(di+i) as described above, where hnage "j" m Figure 45 is in Figure 46, 
the hnage consecutive to hnage "i". For example, m the scan illustrated by block 732 m Figure 
27B, if hnage "i'' is the hnage at block 750, then hnage "j" is the next consecutive hnage (the 
image at block 752). 

30 [0442] Steps 1118, 1120, and 1122 of Figure 45 represent one method of vaUdatmg the 
misalignment correction determined for image "j" m step 1 1 16 of Figure 45. This method of 
validating niisalignment correction is represented m blocks 1177, 1179, 1181, 1190, 1192, and 
1 194 of Figure 47. Another method of validatmg a misaligmnent correction is represented m 
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steps 1 124, 1 126, and 1 128 of Figure 45; and this method is represented in blocks 1 178, 1 180, 
1 182, 1 184, 1 186, and 1 188 of Figure 46. Figure 47 is a schematic flow digram depicting steps 
in a version of the methods shown in Figure 45 of determming a correction for a misalignment 
between two images in which validation is performed using data from two consecutive images. 
5 One embodunent includes using consecutive or near-consecutive images to validate a 

misalignment correction determination, as in Figure 47. Other embodiments comprise using the 
initial image to validate a misalignment correction determination for a given image, as in Figure 
46. 

[0443] In Figure 45, step 1118 represents realigning Gj(x,y), the LoG-filtered data from unage 
10 "j", to match up with Gi(x,y), the LoG-filtered data from image "i", usmg the misalignment 

values dx and dy determined in step 1116. In preferred embodunents, image "j'' is consecutive to 
image "i" in the sequence of images. Here, image "j" is image "i+P' such that Gi(x,y) is aligned 
with Gi+i(x,y) as shown m block 1 177 of Figure 47. Similarly, in Figure 45, step 1 124 represents 
realigning Gj(x,y), the LoG-filtered data from image "j", to match up with Go(x,y), the LoG- 
15 filtered "gold standard" data from the initial image V, using the displacement values sum(dx) 
and sum(dy) determmed in step 1116. Step 1124 of Figure 45 is represented in block 1178 of 
I Figure 46. 

[0444] Step 1 120 of Figure 45 represents comparing corresponding validation cells from 
Gj(x,y) and Gi(x,y) by computing correlation coefficients for each cell. This is represented 

20 schematically in Figure 47 by blocks 1 179, 1181,1 190, 1 192, and 1 194 for the case where j = 
i+1. First, a 128 x 128 pixel central portion of the realigned Gi+i(x,y) is selected, and the 
corresponding 128 x 128 pixel central portion of Gi(x,y) is selected, as shown in blocks 1 179 and 
1181 of Figure 47. An exemplary 128 x 128 pixel validation region 1090 is shown in Figure 
44B. Then, one embodiment comprises computing a correlation coefficient for each of 16 

25 validation cells. An exemplary validation cell from each of the realigned Gi+i(x,y) matrix 1181 
and Gi(x,y) matrix 1 179 is shown in blocks 1 192 and 1 190 of Figure 47. The validation cells are 
as depicted in the 32 x 32 pixel divisions 1092 of the 128 x 128 pixel validation region 1090 of 
Figure 44B. Different embodiments use different numbers and/or different sizes of validation 
cells. Correlation coefficients are computed for each of the 16 cells, as shown in block 1 194 of 

30 Figure 47. Each correlation coefficient is a normalized cross-correlation coefficient as shown in 
Equation 82: 

.■(«.„) = -^J2ii™2= (82) 
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where c'(m^) is the normalized cross-correlation coeflBcient for the validation cell (m^), m is an 
integer 1 to 4 corresponding to the column of the validation cell whose correlation coefficient is 
being calculated, n is an integer 1 to 4 corresponding to the row of the validation cell whose 
correlation coefficient is being calculated, p and q are matrix element markers, Ii[p,q] are 
5 elements of the cell in column m and row n of the 128 x 128 portion of the realigned image 
shown in block 1181 of Figure 47, and l2[p,q] are elements of the cell in column m and row n of 
the 128 X 128 portion of Gi(x,y) shown in block 1 179 of Figure 47. In that figure, p == 1 to 32 
andq= 1 to 32, and the sums shown in Equation 82 are performed over pandq. Thecross- 
correlation coefficient of Equation 82 is similar to an auto-correlation in the sense that a 

10 subsequent image is realigned with a prior image based on the determined misaligmnent 
correction so that, ideally, the aligned images appear to be identical. A low value of cXm,n) 
indicates a mismatching between two corresponding cells. The misaligmnent correction 
determination is then either validated or rejected based on the values of the 16 correlation 
coefficients computed in step 1 194 of Figure 47. For example, each correlation coefficient may 

15 be compared against a threshold maximum value. This corresponds to step 1 122 of Figure 45. 
[0445] Step 1 126 of Figure 45 represents comparing corresponding validation cells firom 
Gj(x,y) and Go(x,y) by computing correlation coefficients for each cell. This is represented 
schematically in Figure 46 by blocks 1 1 80, 1 1 82, 1 1 84, 1 1 86, and 1 1 8 8 for the case where j = 
i+1 . First, a 128 x 128 pixel central portion of the realigned Gi+i(x,y) is selected, and the 

20 corresponding 128 x 128 pixel central portion of Go(x,y) is selected, as shown in blocks 1 182 
and 1 180 of Figure 46. An exemplary 128 x 128 pixel validation region 1090 is shown m Figure 
44B. Then, one embodiment comprises computing a correlation coefficient for each of the 16 
validation cells. An exemplary validation cell firom each of the realigned Gi+i(x,y) matrix 1 182 
and Go(x,y) matrix 1 1 80 is shown in blocks 1 1 86 and 1 1 84 of Figure 46. The validation cells are 

25 as depicted m the 32 x 32 pixel divisions 1092 of the 128 x 128 pixel validation region 1090 of 
Figure 44B. Other embodiments use different numbers of and/or different sizes of validation 
cells. Correlation coefficients are computed for each of the 16 cells, as shown in block 1 1 88 of 
Figure 46. Each correlation coefficient is a normalized "auto'-correlation coefficient as shown 
m Equation 80 above, where Ii[p,q] are elements of the cell in column m and row n of the 128 x 

30 128 portion of the realigned subsequent image shown in block 1 1 82 of Figure 46, and l2[p,q] are 
elements of the cell in column m and row n of the 128 x 128 portion of Go(x,y) shown in block 
1 180 of Figure 46. A low value of c'(m,n) indicates a mismatching between two corresponding 
cells. The misalignment determination is then either validated or rejected based pn the values of 
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the 16 correlation coefficients computed in step 1 188 of Figure 46. This corresponds to step 
1128 of Figure 45. 

[0446] In one embodiment, determinations of misalignment correction and validation of these 
determinations as shown in each of Figure 45, Figure 46, and Figure 47 are performed usmg a 

S plurality of the images in sequence. In one embodiment, determinations of misalignment 
correction and validations thereof are performed while unages are being obtained, so that an 
examination in which a given sequence of unages is obtained may be aborted before all the 
images are obtained. In some embodiments, a misalignment correction is determined, validated, 
and compensated for by adjusting the optical signal detection device obtaining the images. In 

10 certain embodiments, an adjustment of the optical signal detection device is made after each of a 
plurality of images are obtained. In certain embodiments, an adjustment, if required by the 
misalignment correction determination, is made after every image subsequent to the first image 
(except the last image), and prior to the next consecutive image. In one embodiment, a cervical 
tissue scan comprising a sequence of 13 images is performed using on-the-fly misalignment 

15 correction determination, validation, and camera adjustment, such that the scan is completed in 
about 12 seconds. Other embodimrats comprise obtaining sequences of any number of images 
in more or less time than indicated here. 

[0447] Each of steps 1 122 and 1 128 of the embodiment of Figure 45 represents applying a 
validation algorithm to determine at least the following: (1) whether the misalignment oorrection 

20 can be made, for example, by adjusting the optical signal detection device, and (2) whether the 
misalignment correction determined is valid. In an exemplary embodiment, the validation 
algorithm determines diat a misalignment correction cannot be executed during an 
acetowhitening exam conducted on cervical tissue in time to provide sufficiently aligned 
subsequent images, if either of conditions (a) or (b) is met, as follows: (a) di, the displacement 

25 between the cxuxent image "i" and the immediately preceding image "i-l" is greater than 0.55- 
mm or (b) Sum(di), the total displacement between the current image and the first image in the 
sequence, "o", is greater than 2.5-mm. If either of these conditions is met, the spectral scan in 
progress is aborted, and another scan must be performed. If sufficient time remains within the 
optimal time window for obtaining spectral data, a fresh scan may begin immediately after a 

30 previous scan is aborted. Other embodiments may comprise the use of different validation rules. 
In one embodiment, if only condition (a) is met, the system retakes image 'T' while contmuing 
the spectral scan, and if condition (b) is met, the spectral scan is aborted and must be restarted if 
sufficient time remains within the optimal window. 
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[0448] In one embodiment, validation is performed for each detemimation of misalignment 
correction by coimting how many of the correlation coefficients c'r(m,n) shown in Equation 80 
(corresponding to tiie 16 validation cells) is less than 0.5. If this number is greater than 1, the 
scan in progress is aborted. In one embodiment, if there are more than three correlation 
5 coeflScients c'r(m,n) less than 0,35, then the scan is aborted. Other embodiments comprise the 
use of different validation rules. Gradual changes in image features, such as acetowhitening of 
tissue or changes in glare, cause discrepancies which are reflected in the correlation coejBBcients 
of the validation cells, but which do not represent a spatial shift. Thus, in preferred 
embodiments, the validation is performed as shown m Figure 47, where validation cells of 
10 consecutive images are used to calculate the correlation coefficients. In other embodiments, the 
validation is performed as shown in Figure 46, where validation cells of a current image, "i", and 
an initial image of flie sequence, "o", are used to calculate the correlation coeflBcients of 
Equation 80. 

[0449] Figures 48A-F depict a subset of adjusted, filtered images 1200, 1204, 1208, 1212, 

15 1216, and 1220 firom a sequence of images of a tissue with an overlay of gridlines showing the 
validation cells used in validating the determinations of misalignment correction between the 
images, according to an illustrative embodiment of the inventioa By performmg validation 
according to Figure 47, using consecutive images to calculate the correlation coefficients of 
Equation 80, the number of validation cells with correlation coefficient below 0.5 for the 

20 misalignment-corrected images of Figure 48A-F is 0, 1, 0, 0, and 1 for images 1204, 1208, 1212, 
1216, and 1220, respectively. Since none of the images have more than one coefficient below 
0.5, this sequence is successful and is not aborted. There is only a gradually changing glare, seen 
to move within the validation region 1202, 1206, 1210, 1214, 1218, 1222 of each image. In an 
embodiment in which validation is performed as in Figure 46, the numb^ of validation cells 

25 witii correlation coefficient below 0.5 for the misalignment-corrected images of Figure 48A-F is 
3, 4, 5, 5, and 6 for images 1204, 1208, 1212, 1216, and 1220, respectively. This is not a good 
result in this example, since the exam would be erroneously aborted, due only to gradual changes 
in glare or whitening of tissue, not uncompensated movement of the tissue sample. 
[0450] Alternatively, validation cells that are featureless or have low signal-to-noise ratio are 

30 eliminated firom consideration. Those cells can produce meaningless correlation coefficients. 
Featureless cells in a preferred embodiment are identified and eliminated from consideration by 
examining the deviation of the sum squared gradient of a given validation cell fiom the mean of 
tiie sum squared gradient of all cells as shown in Equation 81: 
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IF ssgi(m,n) < Mean[ssg(m,n)] - STD[ssg(m,n)], THEN set c'i(m,n) = LO, (81) 

where c' i(m^) is the correlation of the given validation cell "1", ssgi(nMi) = E S Ii^[p,q], m = 1 
. to 4, n = 1 to 4, Ii [p,q] is the matrix of values of tiie given validation cell "1 p = 1 to 32, q - 1 
to 32, the summations E E are perforaied over pixel markers p and q, Mean[ssg(m,n)] is the 
5 mean of the sum squared gradient of all 16 validation cells, and STD[ssg(m,n)] is the standard 
deviation of the sum squared gradient of the given validation cell "1" from the mean sum 
squared gradient. By setting c' i(m,n) = 1 .0 for the given validation cell, the cell does not count 
against validation of the misalignment correction determination in the rubrics of either step 1 122 
or step 1 128 of Figure 45, since a correlation coefficient of 1.0 represents a perfect matcL 
10 [0451] If an image has large intensity differences between the upper and lower borders and/or 
the left and right borders of the image frame field, LoG filtering may result in "wraparound 
error." A preferred embodiment employs an image blending technique such as '^feathering" to 
smooth border discontinuities, while requiring only a minimal amount of additional processing 
time. 

15 [0452] Figure 49A depicts a sample image 1224 after application of a 9-pixel size [9 x 9] 

Laplacian of Gaussian filter (LoG 9 filter) on an exemplary image from a sequence of images of 
tissue, according to an illustrative embodiment of the invention. The filtered intensity values are 
erroneous at the top edge 1226, the bottom edge 1228, the right edge 1232, and the left edge 
1230 of the image 1224. Since LoG fii^quency domain filtering corresponds to cyclic 

20 convolution ua the space-tune domain, intensity discontinuities between the top and bottom 
edges of an image and between the right and left edges of an image result in erroneous gradient 
approximations. These erroneous gradient approximations can be seen in the dark stripe on the 
right edge 1232 and bottom edge 1228 of the image 1224, as well as the light stripe on the top 
edge 1226 and the left edge 1230 of the image 1224. This often results in a misaligmnent 

25 correction determination that is too small, since changes between the images due to spatial shift 
are dwarfed by the edge effects. A preferred embodiment uses a "feathering" technique to 
smooth border discontinuities and reduce *Svraparound error." 

[0453] Feathering comprises removal of border discontinuities prior to application of a filter. 
In preferred embodiments, feathering is performed on an image before LoG filtering, for 
30 example, between steps 1 100 and 1 102 in Figure 45. In embodiments where LoG filtering is 
performed in the frequency domain (subsequent to Fourier transformation), feathering is 
preferably performed prior to both Fourier transformation and LoG filtering. For two- 
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dimensional image intensity (luminance) functions Ii(x,y) and l2(x,y) that are discontinuous at x 
= Xo, an illustrative feathering algorithm is as follows: 



/K^.;^) = /i(^,>')-/(^^+0.5) and /i(x,;;) = /2(x,>.)-(l-/(.^ + 0.5)). 



0 x<0 
0 x>l 



(82) 



5 where Ii'(x,y) and l2'(x,y) are the intensity.(luminance) functions Ii(x,y) and l2(x,y) after 
applying the feathering algorithm of Equation 82, and d is the feathering distance chosen. The 
feathering distance, d, adjusts the tradeoff between removing wraparound error and suppressing 
image content. 

[0454] Figure 49B depicts the application of both a feathering technique and a LoG filter on 
10 the same unfiltered unage used in Figure 49A. The feathering is performed to account for border 
processing effects, according to an illustrative embodiment of the invention. Here, a feathering 
distance, d,of20 pixels was used. Other embodiments use other values of d. The filtered image 
1234 of Figure 49B does not display uncharacteristically large or small gradient intensity values 
at the top edge 1236, bottom edge 1238, right edge 1242, or left edge 1240, since discontinuities 
15 are smoothed prior to LoG filtering. Also, there is minimal contrast suppression of image detail 
at the borders. Pixels outside the feathering distance, d, are not affected. The use of feathering 
here results in more accurate determinations of misaligmnent correction between two images in a 
sequence of images. 

[0455] Another method of border smoothing is multiplication of unfiltered image data by a 
20 Hamming window. In some embodiments, a Hamming window fimction is multiplied to image 
data before Fourier transformation so that the border pixels are gradually modified to remove 
discontinuities. However, application of the Hanoming window suppresses image intensity as 
well as gradient information near the border of an image. 

[0456] Figure 50A is identical to Figure 49A and depicts flie application of a LoG 9 filter on 
25 an exemplary image firom a sequence of images of tissue according to an illustrative embodiment 
of the invention. The filtered intensity values are erroneous at the top edge 1226, the bottom 
edge 1228, the right edge 1232, and the left edge 1230 of the image 1224, 
[0457] Figure SOB depicts the application of both a Hamming window and a LoG 9 filter on 
the same unfiltered image used in Figure 50A. Hamming windowing is performed to account for 
30 border processing effects, according to an illustrative embodiment of tiie inventioiL Each of the 
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edges 1246, 1248, 1250, 1252 of the image 1244 of Figure 50B no longer show the extreme 
filtered intensity values seen at the edges 1226, 1228, 1230, 1232 of the image 1224 of Figure 
50A. However, there is a greater suppression of image detail in Figure 50B than in Figure 49B. 
Thus, for this particular embodiment, application of the feathering technique is preferred over 
5 application of Hamming windowing. 

[0458] One embodunent includes ronoving cyclic convolution artifacts by zero padding the 
image prior to frequency domain filtering to assure image data at an edge would not a£fect 
filtering output at the opposite edge. This technique adds computational complexity and may 
increase processii^ time. 

10 [0459] Figures 5 1 A-F depict the determination of a misalignment correction between two 
images using methods including the application of LoG filters of various sizes, as well as the 
application of a Hamming window technique and a feathering technique, according to illustrative 
embodimrats of the invention. Image 1254 and image 1256 of Figures 51A-B are consecutive 
images fiom a sequence of images of cervix tissue obtained during a diagnostic exam, each vnih 

15 a pixel resolution of about 0.054-mm. Figures 51C-F depict the application of four different 
image filtering algorithms: (1) Hamming window with LoG 9 filtering, (2) feathering with LoG 
9 filtering, (3) feathering with LoG 21 filtering, and (4) feathering with LoG 31 filtering. Each 
of these algorithms are implemented as part of a misalignment correction determination and 
validation technique as illustrated in Figure 45 and Figure 47, and values of dx and dy between 

20 images 1254 and 1256 of Figures 51A-B are determined using each of Ihe four filtering 

algorithms. For image 1254, each of the four different image filtering algorithms (1) - (4) listed 
above are applied, resulting in images 1258, 1262, 1266, and 1270, respectively, each having 
256 X 256 pixels. The four different imsigt filtering algorithms are also implied for image 1256, 
resulting in images 1260, 1264, 1268, and 1272, respectively, each having 256 x 256 pixels. 

25 Values of (dx, dy) determined using Hamming + LoG 9 filtering are (-7, 0), expressed in pixels. 
Values of (dx, dy) determmed using feathering + LoG 9 filtering are (-2, -10). Values of (dx, dy) 
determined using featiiering + LoG 21 filtering are (-1, -9). Values of (dx, dy) determined using 
feathering + LoG 3 1 filtering are (0, -8). All of the displacement values determined using 
feathering are close in this embodiment, and agree well with visually-verified displacement. 

30 However, in this example, the displacement values determined using Hamming windowmg are 
different from those obtained using tiie o&er three filtering methods, and result in a 
misalignment correction that does not agree well with visually-verified displacement. Thus, for 
this example, fea&ering works best since it does not suppress as much useful image data. 
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[0460] The effect of the filtering algorithm employed, as well as the choice of validation rules 
are examined by applying combinations of the various filtering algorithms and validation rules to 
pairs of sequential images of tissue and determining the number of "true positives" and "false 
positives" identified. A true positive occurs when a bad misaligmnent correction determination 

5 is properly rejected by a given validation rule. A false positive occurs when a good 

misaligmnent correction determination is improperly rejected as a failure by a given validation 
rule. The classification of a validation result as a *true positive" or a "fialse positive" is made by 
visual inspection of the pair of sequential images. In preferred embodiments, whenever true 
failures occur, the scan should be aborted. Some examples of situations where true failures 

10 occur in certain embodiments include image pairs between which there is one or more of the 
following: a large non-translational deformation such as warping or tilting; a large jump for 
which motion tracking cannot compute a correct translational displacement; rotation greater tlian 
about 3 degrees; situations in which a target laser is left on; video system failure such as blur, 
dark scan lines, or frame shifting; cases where the image is too dark and noisy, in shadow; cases 

IS where a vagmal speculum (or other obstruction) blocks about half the image; other obstructions 
such as sudden bleeding. 

[0461] In one embodiment, a set of validation rules is chosen such that true positives are 
maximized and false positives are minimized. Sensitivity and specificity can be adjusted by 
adjusting choice of filtering algorithms and/or choice of validation rules. Table 4 shows the 

20 number of true positives (true failures) and false positives (false failures) determined by a 
validation rule as depicted in Figure 45 and Figure 47 where validation is determined using 
consecutive images. Table 4 shows various combinations of filtering algorithms and validation 
rules. The four filtering algorithms used are (1) Hamming windowing with LoG 9 filtering, (2) 
feathermg with LoO 9 filtering, (3) feathering with LoG 21 filtering, and (4) feathering with LoG 

25 31 filtering. The values, c'(m,n), correspond to the normalized "auto"-correlation coeiBBcient of 
Equation 80 whose value must be met or exceeded iti order for a validation cell to "pass" in an 
embodiment. The "Number Threshold" colimm indicates the maximum number of "failed" 
validation cells, out of the 16 total cells, that are allowed for a misalignment correction 
determination to be accepted in an embodiment If more than this number of validation cells fail, 

30 then the misaUgnment correction determination is rejected. 
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Table 4; True positives and false positives of validation determinations for embodiments using various 
combinations of filtering algorithms and validation rules. 





c*(m,n) 


Number 
Threshold 


TP 


FP 


Hammine LoO 9 


-0.1 


1 


34 


28 


Feathering LoG 9 


-0.1 


3 


19 


17 




0.3 


2 


46 


10 


Feathering LoG 21 


0.35 


3 


52 


4 


Feathering LoG 31 


0.5 


3 


48 


3 



[0462] For the given set of cervical image pairs on which the methods shown in Table 4 were 
applied, feathering performs better than Hamming windowing, since there are more true 
5 positives and fewer false positives. Among different LoG filter sizes, LoG 21 and LoG 3 1 
performs better than LoG 9 for both tracking and validation here. The LoG 21 filter is more 
sensitive to rotation and deformation than the LoG 3 1 filter for these examples. One 
embodiment of the determination and validation of misalignment corrections between 256 x 256 
pixel portions of images of cervical tissue with pixel resolution of about 0.054-mm employs one 

10 or more of the following: (1) use of feathering for image border processing, (2) application of 
LoG 21 filter, (3) elimination of validation cells with low signal-to-noise mtio, and (4) use of 
consecutive images for validation. 

Broadband reflectance arbitration and low-signal masking 
[0463] A tissue characterization system as shown in Figure 1 also may comprise arbitrating 

15 between two or more redundant sets of spectral data as depicted in step 128 of Figure 1 . In one 
embodiment shown m Figure 1, step 128 mcludes arbitrating between two sets of broadband 
reflectance data obtained in step 104 during a spectral scan for each interrogation pomt of a 
tissue sample. Data are obtained at each interrogation point using light incident to the 
interrogation point at two different angles, as depicted in Figure 8. In this way, if only one set of 

20 reflectance data is affected by an artifact such as glare or shadow, the other set can be used in 
tissue classification, for example, in step 132 of Figure L The arbitration step 128 in Figure 1 
determines whether either of the two sets of reflectance spectral data at each pomt is afifected by 
an artifact. Step 128 also determines a smgle set of reflectance data firom each interrogation 
point to be used in tissue classification if at least one of the two sets is acceptably unaffected by 
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an artifact. As used here, artifacts identified in the arbitration step 128 of Figure 1 include, for 
example, both lighting artifacts and obstruction artifacts - such as glare, shadow, blood, mucus, a 
speculum, smoke tube tissue, and/or os tissue. 

[0464] In the embodiment shown in Figure 1, step 128 additionally includes a first-level **hard 

S masking'' of certain interrogation points. For example, interrogation points are considered 

"indeterminate" where values of both sets of reflectance spectral data and/or values of the set of 
fluorescence data are low due to shadow or an obstruction. Additional spectral masks, both hard 
masks and soft masks, are determined in one embodhnent in step 130 of Figure 1. As discussed 
herein, hard-masking of data includes eliminating identified, potentially non-representative data 

10 firom further consideration and identifying the corresponding tissue region as "indeterminate", 
while soft-masking includes applying a weighting function or weighting factor to identified, 
potentially non-representative data so that the importance of the data as a diagnostic indicator of 
a tissue region in a tissue classification algorithm is thereby reduced. A point that is soft-masked 
is not necessarily identified as ""indeterminate" 

15 [0465] The diagram 284 of Figure 8 shows that a misalignment of the probe 142 may create 
conditions where either or both of the top and bottom speculum blades 286 block part or all of 
the illumination path firom either or both of the intersecting upper and lower cones of 
illuminating light 196,198, thereby affectmg the spectral data obtained for the region 250 of the 
tissue sample 194. The speculum blades, or other obstructions present during a spectral scan, 

20 may physically obstruct the region 250 bemg analyzed, or may partially obstruct the light 

illuminating the region 250 causing a shadow. In either case, the spectral data obtained may be 
adversely affected and rendered imusable for characterizing the region of the tissue sample. 
Obtaimng multiple sets of spectral data using illumination fix>m sources at various positions and 
angles improves the chances of obtaining at least one set of spectral data that is not affected by 

25 glare, shadow, and/or obstructions. 

[0466] Figure 52 shows a graph 1276 depicting exemplary mean values of reflectance spectral 
data 1278 as a function of wavelength 1280 for tissue regions affected by glare 1282, tissue 
regions affected by shadow 1284, and tissue regions affected by neither glare nor shadow 1286 
according to an illustrative embodiment of the invention. The reflectance spectral data 1278 

30 represent the fraction of incident light that is reflected from the sample. The graph 1276 shows 
that the reflectance values of a region of tissue affected by glare 1282 are higher at all measured 
wavelengths than the reflectance of a region of tissue not affected by glare 1286. The graph 
1276 also shows that the reflectance values of a region of tissue with illumination partially 
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blocked by a speculum blade such that the region is in shadow 1284, are lower at all measured 
wavelengths than the reflectance of a region of tissue not affected by shadow 1286. The shapes 
of all three curves 1282, 1284, 1286 are different In this example, the data affected by glare or 
shadow may not be usable to determme a condition or characteristic of the region of the sample, 

5 if the data are not representative of the region of the tissue sample. Hence, glare and shadow 
may adversely affect spectral data obtained for a region of a tissue sample. 
[0467] In one embodiment, step 104 of Figure 1 comprises obtaining one fluorescence 
spectrum and two broadband reflectance spectra at each of a plurality of scan locations of the 
sample tissue (interrogation points). Here, a spectrum refers to a collection of spectral data over 

10 a range of wavelengths. In one embodiment method, spectral data are collected over a range of 
wavelengths between 360 and 720 nm in 1 nm increments. In other embodiments, the range of 
wavelengths lies anywhere between about 1 90nm and 1 lOOnm. Here, the two reflectance spectra 
are referred to as the BBl (broadband one) and BB2 (broadband two) spectra. BBl and BB2 
differ in the way that the tissue is illuminated at the time the spectral data are obtained as 

15 described below. In tiie embodunent shown in Figure 6, the probe head 192 has 4 illumination 
sources 222, 224, 226, 228 located circumferentially about the collection optics 200. Two 
sources are above 222, 224 and two are below the horizontal plane 226, 228, as illustrated in the 
second arrangement 212 of Figure 6. The two upper sources are used to obtain BBl spectra and 
the two lower sources are used to obtain BB2 spectra. Since the upper and lower sources 

20 illuminate a region of the tissue sample using light incident to the region at different angles, an 
artifact - for example, or shadow - may affect one of the two reflectance spectra obtained for the 
region, while the otiier reflectance spectrum is unaffected. For example, during acquisition of 
spectral data, the BBl spectrum may be unaffected by an artifact even if the BB2 spectrum is 
adversely affected by the artifact In such a case, BBl spectral data may be used to characterize 

25 the condition of the region of tissue, for example, in step 132 of Figure 1, even flxough the BB2 
data is not representative of the region. In other embodiments, the BBl and BB2 spectra 
comprise one or more other types of spectral data, such as absorbance spectra, adsorption 
spectra, transmission spectra, fluorescence spectra, and/or other types of optical and atomic 
emission spectra. 

30 [0468] Figure 53 shows a graph 1287 depicting mean values and standard deviations of 

broadband reflectance spectral data using the BBl channel light source for regions confirmed as 
being obscured by blood, obscured by mucus, obscured by glare fix)m the BBl source, obscured 
by glare fi:om the BB2 source, or unobscured, according to an illustrative embodiment of the 
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invention. Various sample test points corresponding to regions of tissue from patient scans were 
visually identified as having blood, mucus, or glare present. A sample point was identified as 
having blood present if it was completely covered by blood and if there was no glare. A sample 
point was identified as having mucus present if it was completely covered by mucus and if there 

5 was no glare. A sample point was identified as having glare based on visual evidence of glare 
and large reflectance values in at least one of the two sets of reflectance spectral data (the BBl 
spectrum or the BB2 spectrum). Figure 53 shows the range of BBl reflectance values 1288 for a 
given category of the sample test points which lie within one standard deviation of the mean for 
the category, plotted as a function of wavelength 1290, Figure 53 shows ranges of BBl 

10 reflectance values 1288 for each of the following categories of sample test points: those 

identified as having blood present 1292, those identified as having mucus present 1294, those 
identified as having glare from the BBl illumination source 1296, those identified as having 
glare from the BB2 illumination source 1298, and those identified as unobstructed tissue 1300. 
[0469] Similarly, Figure 54 shows a graph 1301 depicting mean values and standard deviations 

15 of broadband reflectance spectral data using the BB2 channel light source for regions confirmed 
as beir^ obscured by blood 1304, obscured by mucus 1306, obscured by glare &om the BBl 
source 1308, obscured by glare from the BB2 source 1310, or unobscured 1312, according to an 
illustrative embodiment of the invention. Figure 54 shows the range of BB2 reflectance values 
1302 for a given category of the sample test points which lie within one standard deviation of the 

20 mean for the category, plotted as a fimction of wavelength 1290. Figure 54 shows ranges of BB2 
reflectance values 1302 for each of the following categories of sample test points: those 
identified as having blood present 1304, those identified as having mucus present 1306, those 
identified as havmg glare from the BBl illumination source 1308, those identified as having 
glare from the BB2 illummation source 1310, and those identified as unobstructed tissue 1312. 

25 [0470] Figures 53 and 54 show that a region with glare from one illumination source does not 
necessarily have high reflectance values corresponding to data obtained using the other 
illumination source. For example, in Figure 53, the range of BBl reflectance values 1288 of 
points with visual evidence of glare from the BB2 source 1298 is similar to the range of BBl 
reflectance values 1288 of unobstructed tissue 1300. Similarly, in Figure 54, the range of BB2 

30 reflectance values 1302 of points demonstrating glare from the BBl source 1308 is similar to the 
range of BB2 reflectance values 1302 of unobstructed tissue 1312. Therefore, one of the two 
sets of reflectance spectral data may be usefiil in characterizing the tissue even if the other of the 
two sets is corrupted by an artifact, such as glare. 
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[0471] It may also be desirable to determine spectral characteristics caused by various artifacts 
so that data correspondii^ to a region affected by a given artifact may be identified or to 
determine a spectral characteristic of an artifact based on the spectral data itself, without having 
to rely on other visual evidence of a given artifact. In order to detennine these spectral 
characteristics, an embodunent of the invention comprises using spectral data known to be 
affected by a given artifact based on visual evidence, as well as spectral data known not to be 
affected by an artifact. Techniques that may be used to identiiy spectral characteristics and/or to 
develop classification rules determining whether given data are affected by an artifact include, 
for example, discriminant analysis (linear, nonlinear, multivariate), neural networks, principal 
component analysis, and decision tree analysis. One embodiment comprises deteraiining a 
particular wavelength that gives the greatest difference between the artifact-affected spectral data 
(the outlier) and spectral data firom conesponding nearby tissue that is known to be unaffected by 
the artifact (the tissue). Alternatively, the embodiment comprises determiiung a wavelength that 
gives the largest difference between the outlier and the tissue, as weighted by a measure of 
variability of the data. In one embodiment, this method locates where the difference between the 
mean reflectance for the outlier and the tissue is at a maximum relative to the difference between 
the standard deviations for the outlier data and the tissue data. In one embodunent, the method 
determines a maximum value of D as a function of wavelength, where D is the difference given 
in Equation 83 below: 

where ti(BB(X))ouitter is the mean of a set of reflectance spectral data at wavelength X known to be 
affected by a given artifact, ii{BB{X))Ti3sue is the mean of a set of reflectance spectral data at 
wavelength A that is known not to be affected by the artifact, o{BB{X))outiier is the standard 
deviation of the set of reflectance spectral data at wavelength X known to be affected by the 
given artifact, and cj(BB{X))rissue is the standard deviation of the set of reflectance spectral data at 
wavelength A known not to be affected by the given artifact. 

[0472] Figure 55 shows a graph 1313 depicting the weighted difference 1314 between the 
mean reflectance values of glare-obscured regions and unobscured regions of tissue as a fimction 
of wavelength 1316, according to an illustrative embodiment of the invention. The weighted 
difference 1314 is as given in Equation 83. For tiie data sets used in Figure 55, the wavelength 
providing the maxunum value 13 1 8 of D in Equation 83 is about 420 nm. Thus, exemplary 



wo 2004/095359 



PCT/US2004/011820 



-115- 

spectral characteristics identifiable with this set of glare-obscured "outlier" data include the 
reflectance spectral data at around 420nm, and any deviation of this data from reflectance 
spectral 'tissue" data for unobscured regions of correspondingly similar tissue at around 420nnL 
This embodunent uses reflectance spectral data. O&er embodiments may use other types of 

5 spectral data, mcluding fluorescence data. 

[0473] Figure 56 shows a graph 1319 depicting the weighted difference 1314 between the 
mean reflectance values of blood-obscured regions and unobscured regions of tissue as a 
function of wavelength 1316, according to an illustrative embodiment of the invention. The 
weighted difference is as given in Equation 83. For the data sets used in Figure 56, the 

10 wavelength providing the maximum value 1320 of D in Equation 83 is about 585 nm. 

{0474] Thus, exemplary spectral characteristics identifiable with this set of blood-obscured 
"outlier" data include the reflectance spectral data at about 585nm, and any deviation of this data 
from reflectance spectral "^ssue" data for imobscured regions of correspondingly similar tissue 
at about 585nm. This embodiment uses reflectance spectral data. Other embodiments may use 

15 other types of spectral data, including fluorescence spectral data. 

[0475] Figure 57 shows a graph 1321 depicting the weighted difference 13 14 between flie 
mean reflectance values of mucus-obscured regions and unobscured regions of tissue as a 
function of wavelength 1316, according to an illustrative embodiment of the invention. The 
weighted difference is as given in Equation 83. For the data sets used in Figure 57, the 

20 wavelength providing the maximum value 1322 of D in Equation 83 is about 577 nm. Thus, 
exemplary spectral characteristics identifiable with this set of mucus-obscured "outlier" data 
include the reflectance spectral data at about 577 nm, and any deviation of this data flx>m 
reflectance spectral ^Hissue" data for unobscured regions of correspondingly similar tissue at 
about 577 nm. This embodiment uses reflectance spectral data. Other embodiments may use 

25 other types of spectral data, including fluorescence spectral data. 

[0476] One illustrative embodiment comprises determining two wavelengths where the ratio of 
spectral data at the two wavelengths is most different for the artifact-affected spectral data (the 
"outlier") and spectral data from corresponding nearby tissue that is known to be unaffected by 
the artifact (the "tissue"). Alternatively, the method comprises determining two wavelengths 

30 where the ratio of spectral data at the two wavelengths weighted by a measure of variability is 
most different for the outlier data and the tissue data. In one embodiment, the method comprises 
determining a maximum value of D as a function of wavelength, where D is the difference given 
in Equation 84 below: 
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Wb{^)ibb{x%,,„-m(bb(x)ibb(2:\^ 

yla'{BB{X)/BB(X'))o^^a\BB(z)/BB(Jf^ ' ^ ^ 

where fi(BB(X)/BB(X '))outiier is the mean of the ratios of reflectance at wavelength A and 
reflectance at wavelength i ' for a set of reflectance spectral data known to be affected by a given 
artifact, n{BB{X)IBB{X '))nssue is the mean of the ratios of reflectance at wavelength X and 
5 reflectance at wavelength A ' for a set of reflectance spectral data that is known not to be affected 
by the given artifect, a(BB(X)/BB(X'))outiier is the standard deviation of the ratios of reflectance at 
wavelength A and reflectance at wavelength A' for a set of reflectance spectral data known to be 
affected by the given artifact, and o(55(A)/55(A '))77««e is the standard deviation of the ratios of 
reflectance at wavelengtti A and reflectance at wavelength A ' for a set of reflectance spectral data 

10 known not to be affected by the given artifact 

[0477] Figure 58 shows a graph 1323 depicting a ratio of the weighted differences 1324 
between the mean reflectance values of glare-obscured regions and unobscured regions of tisstie 
at two wavelengths, a numerator wavelength 1326 and a denominator wavelength 1 328, 
according to an iUustrative embodunent of flie mvention. The weighted difference 1324 is as 

15 givoi in Equation 84. For the data sets used in Figure 58, the two wavelengths providing the 
ma x imum value of D in Equation 84 are about 401 nm (numwator) and about 404 run 
(denominator). Thus, exemplary spectral charactMistics identifiable witii this set of glare- 
obscured "ouflier" data include the ratio of reflectance spectral data at about 401nm and the 
reflectance spectral data at about 404nm, as well as any deviation of this ratio from those of 

20 corresponding regions of similar but unobscured tissue. This embodiment uses reflectance 
spectral data. Other embodiments may use other types of spectral data, including fluorescence 
data. 

[0478] Figure 59 shows a gnqph 1325 depicting a ratio of the weighted differences 1324 
between the mean reflectance values of blood-obscured regions and unobscured regions of tissue 

25 at two wavelengths, a numerator wavelengfli 1 326 and a denominator wavelength 1328, 

according to an illustrative embodiment of the invention. The weig^hted difference is as given in 
Equation 84. For the data sets used m Figure 59, the two wavelengtiis providing the maximum 
value of D in Equation 84 are about 595 nm (numerator) and about 718 nm (denominator). 
Thus, an exemplary spectral characteristic identifiable with this set of blood-obscured "outiier" 

30 data includes the ratio of the reflectance spectral data at about 595nm and the reflectance spectral 
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data about 71 8nm. This embodiment uses reflectance spectral data. Other embodiments may 
use other types of spectral data, including fluorescence data, 

[0479] Figure 60 shows a graph 1327 depicting a ratio of the weighted differences 1324 
between the mean reflectance values of mucus-obscured regions and unobscured regions of 
tissue at two wavelengths, a numerator wavelength 1326 and a denominator wavelength 1328, 
according to an illustrative embodiment of the invention. The weighted difference is as given in 
Equation 84. For the data sets used in Figure 60, the two wavelengths providing the mayimu m 
value of D in Equation 84 are about 545 nm (numerator) and about 533 nm (denominator). 
Thus, an exemplary spectral characteristic identifiable with this set of mucus-obscured "outlier" 
data includes the ratio of the reflectance spectral data at about 545mn and the reflectance spectral 
data at about 533nm. This embodiment uses reflectance spectral data. Other embodiments may 
use other types of spectral data, including fluorescence data. 

[0480] Another type of lighting artifact which may obscure spectral data is shadow, which may 
be caused, for example, by an obstruction blockmg part of the light from an illumination source 
on the optical probe 142 of the embodiment apparatus. It may be important to differentiate 
between glare and shadow, so that spectral data representing unobstructed tissue can be properly 
identified. In an embodiment, broadband reflectance is expressed as the intensity of light 
diffusely reflected from a region of the tissue, It, over the intensity of incident light, I©, at the 
region. When glare is measured in addition to light diffusely reflected from the tissue, a 
percentage of the original intensity of incident light is included in the tissue reflectance 
measurement, so that the "reflectance" readmg of a region of a sample experiencing glare, Rg(3l), 
may be expressed as m Equation 85: 

W = (IA) + aW)/Io(X) , (85) 
where a is a real number between 0.0 and 1 .0; li(X) is tfie intensity of light diffusely reflected 
from the region of tissue at wavelength A., and lo(X) is the intensity of light incident on the region 
oftiie sample at wavelength A.. The intensity of the specularly-reflected light is aIo(A,). When 
the region of the sample is shadowed, only a portion of the incident intensity reaches the region. 
Thus, the **reflectance" reading of a region of a sample experiencing shadow, Rs(^), may be 
expressed as in Equation 86: 

Rs(A.) = pit(A,)/Io(^). (86) 
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where p is a real number between 0,0 and 1.0; It(X) is the intensity of light at wavelength 

X diffiisely reflected from the region of tissue with an incident light intensity of Io(X), and Io(A,) is 

the intensity of light at wavelength X that would be incident on the region of the sample if 

unshadowed. 

S [0481] In one embodhnent, the arbitration in step 128 of Figure 1 comprises determining if 
only one set of a pair of sets of spectral data is afTected by a lighting artifact, such as glare or 
shadow, each set having been obtained using light incident on the sample at a unique angle. If it 
is determined that only one set of a pair of sets of spectral data is affected by the artifact, then the 
other set of spectral data may be used in the determination of a characteristic of the region of the 

10 sample, for example. In one embodiment, it is determined that there is evidence of a lighting 
artifact in the spectral data. Such evidence may be a large difference between the reflectance 
measurements of the two sets of spectral data. If such evidence exists, then one of the 
reflectance measurements will either be Rg or Rs, as given by Equation 85 and Equation 86. In 
cases where members of only one set are affected by a lighting arti&ct, the remaining set of 

15 reflectance measurements may be expressed as R, the intensity of light dififiisely reflected from 
tihe region of the tissue, It, divided by the intensity of light incident on the region of the tissue, Iq. 
In an embodiment method, the larger of the two reflectance measurements corresponding to a 
given wavelength is divided by the smaller. In cases where only one of the sets is affected by a 
lightmg artifact, the resulting quotient will be either Rg/R, which is equal to HaloiX)/lt(X\ or 

20 R/Rs, which is equal to the constant, 1/p. If glare is present, the value of the quotient will depend 
on wavelength and the plot of the quotient as a function of wavelength should look like an 
inverted unobstructed tissue broadband signal because of the aIo(X)/It(A,) term. If shadow is 
present, the plot of the quotient should be constant across the spectrum. 
[0482] Figure 61 shows a graph 1332 depicting as a fimction of wavelength 1336 mean values 

25 and confidence intervals of a ratio 1334 of BB 1 and BB2 broadband reflectance spectral values 
(larger value divided by smaller value) for regions confirmed as being either glare-obscured or 
shadow-obscured tissue, according to an illustrative embodiment of the invention. The shadow 
pomts 1338 yield a nearly constant value, while the glare pomts 1340 vary over tiie range of 
wavelength 1336 in a manner that resembles the inverse of unobstructed tissue reflectance. 

30 Thus, Figure 6 1 illustrates an embodiment in which it is determined whether only one set of a 
pair of sets of spectral data is affected by either glare or shadow, such that the other set is 
unaffected by glare or shadow and may be used to determine a characteristic of the tissue, for 
example. In an embodiment, the method comprises differentiating between glare and shadow by 
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observing the steep slope of glare-affected reflectance spectral measurements between about 
577nm and 599nm5 for example, compared to the nearly flat slope of shadow-afifected 
reflectance spectral measurements at those wavelengths, as seen in Figure 61. 
[0483] In one embodiment, the arbitration in step 128 of Figure 1 includes applying and/or 

S developing spectral artifact classification rules (metrics) using spectral data, mcluding one or 
more sets of fluorescence and broadband reflectance data obtained using light at one or more 
angles. In one embodiment, one set of fluorescence data and two sets of reflectance data are 
obtained from a given region of a tissue sample (interrogation point), where each of the two sets 
of reflectance data are obtained using light incident on the region at a different angle. These 

10 metrics determine what data is representative of a given region of tissue. By varying the metrics, 
desired levels of sensitivity and selectivity of a resulting tissue characterization using tissue- 
representative data may be achieved. 

[0484] The following metrics are applied in one embodiment of the arbitration in step 128 of 
Figure 1 and were determined using the embodiments discussed above. These metrics were 

15 developed using one set of fluorescence data and two sets of reflectance data, BB 1 and BB2, for 
samples of cervical tissue. Other embodhnents use other combinations of spectral data sets. 
Each of the two sets of reflectance data used in the following metrics were obtained using light 
incident to a region of a sample at different angles. An embodiment of the invention uses any or 
all of the metrics listed below to determine if any set of data should be eliminated from use in 

20 determining a characteristic of a region of tissue, due to the presence of a spectral artifact. In an 
embodiment of the invention, wavelengths within a range of the wavelengths shown below are 
used. In one embodiment, this range about the wavelengths is about d:10nm. In an embodiment 
of Ihe invention, only certain parts of the metrics shown below are used. In one embodiment, 
only a portion of a given set of spectral data are eliminated, not the entire set. In one 

25 embodiment, BBl and BB2 reflectance data are obtained, but fluorescence data is not Here, 
''elimuiate data*' means to eliminate data from consideration in an analysis, for example, an 
analysis to determine a condition of a region. It is possible to change sensitivity and selectivity 
of a tissue diagnostic algorithm by varying the metrics below, for instance by changing one or 
more of the threshold constants. Such variations are within an embodiment of this invention. 

30 The metrics for one exemplary embodiment are as follows: 
Glare Metric #1; Eliminate BBl data IF; 
L {BB1(419)> 0.25 AND BB1(699)> 0.51} OR BB1(529)/BB1(543)<L0; 
OR n. MaK{|ABB|/avgBB}(370-710)> 0.25 AND BB1(419)> 0.18 
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AND BB 1(699) > 0.51 AND 
{BB1(576)/BB2(576)}/{BB1(599)/BB2(599)}> 1.1 ; 
OR ra.Max{|ABB|/avgBB}(370-710)>0.4AND 

{BB1(576)/BB2(576)}/{BB1(599)/BB2(599)}>1.1 AND BB2(699) > 0.3. 

Glare Metric #2; Eliminate BB2 data IF; 
L {BB2(419)> 0.25 AND BB2(699)> 0.51} OR 
BB2(529yBB2(543)<1.0; 
OR n. Max{|ABB|/avgBB}(370-710) > 0.25 AND BB2(419)> 0.18 
' AND BB2(699)> 0.51 AND 

{BB2(576)/BB1(576)}/{BB2(599)/BB1(599))>1.1; 
OR in. Max{|ABB|/avgBB}(370-710)>0.4AND 

{BB2(576)/BB1(576)}/{BB2(599)/BB1(599)}>1.1 AND BB1(699)> 
0.3. 

Shadow Metric #1; Eliminate BBl data IF! 

L BB2(499)>BB1(499) AND Max{|ABB|/avgBB}(370-710) > 0.25 AND BB1(499) 
0.05; 

OR n. Max{|ABB|/avgBB}(370-710)>0.5AND 

{BB1(576)/BB2(576)}/{BB1(599)/BB2(599)}<1.1 AND 
BB2(576)>BB1(576) AND BB1(419) < 0.2. 

Siiadow Metric it2: Eliminate BB2 data DF; 

I. BB1(499)>BB2(499) AND Max{|ABB|/avgBB}(370-710) > 0.25 AND BB2(499) 
0.05; 

OR n. Max{|ABB|/avgBB}(370-710)>0.5AND 

{BB2(576)/BB1(576)}/{BB2(599)/BB1(599)}<1.1 AND 
BB1(576)>BB2(576) AND BB2(419) < 02. 



wo 2004/095359 



PCT/US2004/011820 



-121- 

Low Signal; Eliminate BBl, BB2. and Fl data IF; 
I. Fl(479) < 3 .5 counts/jJ (where mean fluorescent intensity of 
normal squamous tissue is about 70 counts/|J at about 4S0nm); 
OR n. BB1(499)< 0.035 & BB2(499)< 0.035. 

5 where BBIPQ is the BBl reflectance spectrum measurement at wavelength X, BB2CX) is tiie 
BB2 reflectance spectrum measurement at wavelength X, Max{|ABB|/avgBB}(370-710) 
indicates the maximum of {the absolute value of the difference between the BBl and BB2 
reflectance spectrum measurements divided by the average of the BBl and BB2 measurements at 
a given wavelength} over the range of about 370 to 710nm, and F1(X) is the fluorescence 

10 spectrum measurement at wavelength X. The following are notes regarding the Metrics listed 
above and apply to a preferred embodiment, subject to the variations described above: 
Glare Metric #1 and Glare Metric #2: 

Level I: Broadband measurements are generally greater than about 0,25 at about 419nm 
and greater than about 0.5 1 at about 699 nm only when there is glare in the channel (i.e. 

15 BBl or BB2). The lack of a downward slope between about 499 and about 543 nm is 

also a strong indication that the broadband measurements are affected by glare. 
Level n: Large percentage differences in the broadband measurements combined with 
higher than average reflectance at about 419 nm and about 699 nm also indicates the 
presence of glare. The presence of a slope when the broadband measurements at about 

20 576 nm and about 599 nm are divided is further confirmation that glare is present. 

Level UL A maximum broadband percent difference that is larger than about 0.4 
indicates that there is a lighting artifact present. The presence of a slope when the 
broadband measurements at about 576 and about 599 nm are divided and an off-channel 
broadband greater than about 0.3 at about 699 nm reveals that the lighting artifact is due 

25 to glare instead of shadow. 

If a point is identified as glare in one channel, then subsequently identified as glare in 
both channels, both broadband measurements shotild be eliminated. 
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Shadow Metric #1 and Shadow Metric #2; 

Level I: Broadband measurements that are shadowed generally will have a large percent 
difference between BBl and BB2 and a low reflectance at about 499 nm. 
Level n: A maximum broadband percent difference that is larger than about 0.5 indicates 
tiiat there is a lightmg artifact present. Lacking a large slope when the broadband 
measurements at about 576 and about 599 nm are divided and an off-channel broadband 
less than about 02 at about 419 nm reveals that the point is shadow instead of glare. 

Cases where both BB and Fl measuremeras should be eliminated: 

Low Signal; 

Broadband measurements lower than about 0.035 at about 449 nm or fluorescence 
measurements lower than about 3.5 at about 479 nm indicate that the measurements are 
not coming from tissue, but mther from blood, the os, smoke tube, speculum, or anotiier 
obstruction. Sites with significant shadowmg in both broadband channels are also 
identified with this metric. Because of the uncertainty of the tissue being measured, the 
reflectance and fluorescence data from that point are assumed invalid, regardless of 
whether it was identified by fluorescence or the broadband channels. 

The low signal metric acts as a hard mask because it eliminates a qualifying interrogation 
point from consideration by tiie classifier or the other masks, such as the spectral masks 
in step 130 of Figure 1 . The low signal metric acts as a hard mask, for example, for 
points that have shadowing in both BBl and BB2. 

[0485] The metrics used in this embodiment of step 128 of Figure 1 include a low signal 
metric, which detects spectral data affected by obstruction artifacts such as blood, a speculum, a 
smoke tube, or other obstruction. This metric also identifies regions where both sets of 
broadband reflectance data are affected by shadow. These were combined into one low signal 
metric in this embodiment, since regions affected by these artifacts exhibit similar 
characteristics, such as low fluorescence and low broadband reflectance measurements. 
[0486] Figure 62 shows a graph 1342 depicting broadband reflectance 1344 as a fimction of 
wavelength 1346 for the BBl channel 1348 and the BB2 channel 1350 measurements for a 
region of tissue where the BBl data is affected by glare but the BB2 data is not, according to an 
illustrative embodiment of the invention. The glare leads to a higher value of reflectance 1344 
than that of surrounding unaffected tissue. By applyii^ the metrics listed above in step 128 of 
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Figure 1, it is determined that the exemplary BBl set of spectral data shown in Figure 62 is 
affected by glare and is thus not suitably representative of this region of the tissue sample. 
Applying the metrics of step 128 also determines that the BB2 set of spectral data is potentially 
representative of this region of the sample (unaffected by an artifact), since it is not eliminated. 
5 One embodiment comprises using this representative data in step 132 of Figure 1 to determine a 
condition of this region of the sample, for example, the state of health. 
[04871 Figure 63 shows a graph 1351 depicting broadband reflectance 1344 as a function of 
wavelength 1346 for the BBl channel 1352 and the BB2 channel 1354 broadband reflectance 
spectral data for a region of tissue where the BB2 data is affected by shadow but the BB 1 data is 

10 not, according to an illustrative embodiment of the invention. The shadow leads to a lower value 
of reflectance 1344 than that of surrounding unaffected tissue. By applying the metrics listed 
above in step 128 of Figure 1, it is determined that the exemplary BB2 set of spectral data shown 
in Figure 63 is affected by shadow and is therefore not suitably representative of this region of 
the tissue sample. Applymg the metrics of step 128 also leads to the determination that the BB 1 

15 set of spectral data is potentially representative of this region of the sample, since flie BBl set of 
data is not eliminated. One embodiment comprises using this representative data in step 132 of 
Figure 1 to determine a condition of this region of the sample, for example, the state of health. 
[0488] Figure 64 shows a graph 1358 depicting broadband reflectance 1360 as a function of 
wavelength 1362 for the BBl channel 1364 and the BB2 channel 1366 measurements for a 

20 region of tissue that is obscured by blood, according to an illustrative embodiment of the 

invention. By applying the metrics listed above, it is determined that blood is present, and that 
both the BBl and the BB2 sets of spectral data are considered unrepresentative of this region of 
the tissue sample. 

[04891 Figure 65 shows a graph 1367 depicting broadband reflectance 1360 as a function of 
25 wavelength 1362 for the BBl channel 1368 and the BB2 channel 1370 measurements for a 
region of tissue that is unobscured, according to an illustrative embodhnent of the invention. 
Applying this method determines that neither set of spectral data is affected by an artifact, and, 
therefore, either is representative of the tissue sample. One embodiment comprises using an 
average value 1372 of the BBl and BB2 measurements at each wavelength to represent the 
30 region of the tissue sample in determining a condition of this region, for example, the state of 
health of the region, in step 132 of Figure 1. 

[0490] Application of the metrics listed above was performed using various tissue types to 
verify the sensitivity and specificity of flie metrics. While, in one embodiment, it is imdesirable 
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to eliminate good spectral data of normal tissue, it is worse to eliminate good spectral data of 
diseased tissue, particularly if it is desired to use the data in the classification of the state of 
health of a region of tissue. The following tissue types were used in the verification: tt-132 
(metaplasia by impression), tt-155 (normal by unpression), tt-1 17 (blood), NEDpath (no 

5 evidence of disease confirmed by pathology), and cin23all (CIN 2/3 diseased tissue). Table 5 
shows the number of points (regions) corresponding to each of these tissue types, the 
determinations firom tiie metrics listed above for these points, and the mmiber of points where 
one set of broadband reflectance spectral data were eliminated, where both sets of broadband 
reflectance spectral data were eliminated, and where both reflectance and fluorescence spectral 

10 data were eliminated. 



Table 5; Verification of Metrics 



Tissue Type 


cin23all 


nedpath 


tt-117 


tt-1 32a 


tt-16S 


Total pts. 


477 


919 


175 


5000 


2016 


Low Signal 


2 


14 


126 


2 


0 


Glare in BB1 


7 


30 


4 


122 


26 


Glare In BB2 


9 


40 


9 


134 


16 


Glare in both 


3 


5 


1 


15 


5 


Shadow in BB1 


47 


35 


4 


165 


132 


Shadow in BB2 


16 


37 


24 


359 


32 


One BB Removed(%) 


16.6 


15.5 


23.4 • 


15.6 


10.2 


Both BB Removed(%) 


1.05% 


2.07% 


72.57% 


0.34% 


0.25% 


Fi Renrioved(%) 


0.42 


1.52 


72.00 


0.04 


0.00 



[0491] For the regions (points) corresponding to CIN 2/3 diseased tissue, no broadband 
reflectance measurements were unnecessarily eliminated from the set using the above metrics. 
15 The points identified as being low signal were all located on the os. All points that were 

identified by the metric as shadow were verified as being correct, and only one point identified 
as glare was incorrect 

[0492] For the nedpath points (no evidence of disease), only two tissue points were 
umiecessarily eliminated after being misidentified as mucus. A point that vm actually dark red 
20 tissue with glare was mcorrectly identified as shadow in BB2. The points that were identified as 
glare were verified as being correct. 

[0493] Out of the 175 blood points, 126 were identified as being low signal. The glare points 
and shadow points were accurate. 

[0494] Out of the 5000 points in the metaplasia by impression group, there were no valid tissue 
25 points lost. The data set was improved by eliminating about 800 readings of points affected by 
either glare or shadow. 

[0495] Out of the 2016 normal by impression pomts, no measurements were unnecessarily 
removed from the set 



wo 2004/095359 



PCTAJS2004/011820 



- 125 - 

[0496] Figure 66 shows a graph 1 374 depicting the reduction in the variability of broadband 
reflectance measurements 1376 of CIN 2/3-confirmed tissue produced by filtering (eliminatii^ 
non-representative spectral data) using the metrics of step 128 in Figure 1 described above, 
accorcUng to an illustrative embodiment of the invention. The graph 1374 depicts mean values 

5 and standard deviations of broadband reflectance spectral data before and after filtering. 

[0497] Figure 67 shows a graph 1378 depictmg the reduction in the variability of broadband 
reflectance measurements 1376 of tissue classified as '^no evidence of disease confirmed by 
pathology" produced by filtering using the metrics described above, according to an illustrative 
embodiment of the invention. The graph 1378 depicts mean values and standard deviations of 

10 broadband reflectance spectral data before and after filtering. 

[0498] Figure 68 shows a graph 1380 depicting the reduction in the variability of broadband 
reflectance measurements 1376 of tissue classified as "'metaplasia by impression" produced by 
filtering using the metrics described above, according to an illustrative embodiment of the 
invention. The graph 1380 depicts mean values and standard deviations of broadband 

15 reflectance spectral data before and after filtering. 

[0499] Figure 69 shows a graph 1382 depicting the reduction in the variability of broadband 
reflectance measxirements 1376 of tissue classified as "normal by impression" produced by 
filtering using the metrics described above, according to an illustrative embodiment of the 
invention. The graph 1382 depicts mean values and standard deviations of broadband 

20 reflectance spectral data before and after filtering. 

[0500] Figure 70A depicts an exemplary image of cervical tissue 1388 divided into regions for 
which two types of reflectance spectral data and one type of fluorescence spectral data are 
obtained, according to one embodiment of the invention. Figure 70B is a representation 1398 of 
the regions depicted in Figure 70A and shows the categorization of each region using the metrics 

25 in step 128 of Figure 1. The black-highlighted sections 1390 of tiie image 1388 in Figure 70A 
correspond to points (regions) that had both reflectance measurements elhninated by application 
of the embodiment method. Many of the lower points 1392, as seen in both Figures 70 A and 
70B, are in shadow because the speculum obstructs the view of one of the channels. Glare is 
correctly identified prominenfly at the upper one o'clock position 1394. Since there are blood 

30 points on the shadowed section, some are labeled blood (low signal) and others are treated as 
shadow. 

[0501] Figure 71 A depicts an exemplary image of cervical tissue 1402 divided into regions for 
which two types of reflectance spectral data and one type of fluorescence spectral data are 
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obtained, according to one embodiment of the invention. Figure 71B is a representation 1406 of 
the regions depicted in Figure 71 A and shows the categorization of each region using the metrics 
in step 128 of Figure L Figures 71 A and 71B show an example of a cervix that has a large 
portion of the lower half 1404 affected by shadow. However, only one of the sets of reflectance 
spectral data (BB2) is affected by the shadow artifact The BBl reflectance spectral data is not 
affected by shadow. Applying the metrics above, the BBl data are used to describe these 
regions, while the BB2 data are eliminated fiom consideration. The accuracy of tissue 
characterization usmg the reflectance measurements should be improved significantly for this 
patient using the arbitration metrics of step 128 of Figure 1, since the more accurate broadband 
measurements will be used in later characterization steps instead of simply averaging the two 
broadband measurements, which would skew the measurements due to a lighting artifact. 
[0502] Figure 72A depicts an exemplary image of cervical tissue 1410 divided into regions for 
which two types of reflectance spectral data and one type of fluorescence spectral data are 
obtained, according to an illustrative embodiment of flie invention. Figure 72B is a 
r^resentation 1416 of the regions depicted in Figure 72A and shows the categorization of each 
region using the metrics in step 128 of Figure 1 . Figures 72A and 72B show an image with a 
portion 1412 that is shadowed and off of the cervix. Due to an obstruction &om the smoke tube 
in the upper part of the image, there are many low signals. Even though much of the cervix is 
shadowed in BBl 1414, there are still some BB2 and fluorescence readings usable in later tissue 
classification steps. 

Classification svstem overview 
[0503] The tissue characterization system 100 of Figure 1 combines spectral data and image 
data obtained by the instrument 102 to characterize states of health of regions of a tissue sample. 
In one embodiment, the spectral data are first motion-tracked 106, preprocessed 1 14, and 
arbitrated 128 before being combined with image data in step 132 of Figure 1. Likewise, in one 
embodiment, the image data are first focused 122 and calibrated 124 before being combined with 
spectral data in step 132 of Figure 1. Each of these steps are discussed in more detail herein. 
[0504] Figure 73 shows how spectral data and image data are combined in the tissue 
characterization system of Figure 1, according to one embodiment. The block diagram 1420 of 
Figure 73 depicts steps in processing and combining motion-tracked 106, preprocessed 114, and 
arbitrated 128 spectral data with focused 122, calibrated 124 hnage data to determine states of 
health of regions of a tissue sample. After preprocessing 1 14, spectral data from each of the 
interrogation points (regions) of the tissue sample are arbitrated in step 128 of Figure 73. In the 
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embodiment shown, a fluorescence spectrum, F, and two broadband reflectance spectra, BBl 
and BB2, are used to determine one representative reflectance spectrum, BB, used along with the 
fluorescence spectrum, F, for each interrogation point. This is depicted in Figure 73 as three 
heavy arrows representing the three spectra - BBl, BB2, and F - entering arbitration block 128 
S and emerging as two spectra - BB and F. Block 128 of Figure 73 also applies an initial low- 
signal mask as a first pass at identifying obscured interrogation points, discussed previously 
herein. 

[0505] In tiie embodiment of Figure 73, the arbitrated broadband reflectance spectrum, BB, is 
used m the statistical classification algorithm 134, while both the broadband reflectance 

10 spectrum, BB, and the fluorescence spectrum, F, as well as the image data, are used to determine 
heuristic-based and/or statistics-based metrics, or "masks", for classifying the state of health of 
tissue at interrogation points. Masking can be a means of identifying data that are potentially 
non-representative of the tissue sample. Potentially non-representative data includes data that 
may be affected by an artifact or obstruction such as blood, mucus, fluid, glare, or a speculum. 

IS Such data is either hard-masked or soft-masked. Hard-maskmg of data includes identifying 
interrogation points at which the data is not representative of unobscured, classifiable tissue. 
This results in a characterization of "Indeterminate" at such an interrogation point, and no furtiier 
computations are necessary for that point. Soft-masking mcludes applying a weighting function 
or weighting factor to identified, potentially non-representative data. The weighting is taken into 

20 account during calculation of disease probability and may or may not result in an indeterminate 
diagnosis at the corresponding tissue region. Soft-masking provides a means of weighting 
spectral and/or image data according to the likelihood that the data is representative of clear, 
unobstructed tissue in a region of interest. In the embodiment shown in Figure 73, both hard 
masks and soft masks are determined using a combination of spectral data and image data. 

25 Furthermore, the masks of Figure 73 use spectral and image data to identify interrogation points 
that are not particularly of interest in the exam, such as the vaginal wall, smoke tube tissue, the 
OS, or tissue outside the region of interest. 

[0506] In addition to determinmg data that are potentially non-representative of regions of 
interest, the masks shown in Figure 73 also include masks that determine where the data is 
30 highly indicative of necrotic tissue or disease-free (NED) tissue. It has been discovered that 
necrotic tissue and disease-firee tissue are often more predictably determined by using a heuristic 
metric instead of or in combination with a statistical classifier than by using a statistical classifier 
alone. For example, one embodiment uses certain values firom fluorescence spectra to determine 
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necrotic regions, since fluorescence spectra can indicate the FAD/NADH component and 
porphyrin component of necrotic tissue. Also, an embodiment uses prominent features of 
fluorescence spectra indicative of normal squamous tissues to classify tissue as '*NED" (no 
evidence of disease) in the spectral mask. 
S [0507] Identifying necrotic and NED regions at least partially by using heuristic metrics allows 
for the development of statistical classifiers 134 that concentrate on differentiating tissue less 
conducive to heuristic classification - for example, statistical classifiers that differentiate high 
grade cervical intraepithelial neoplasia (i.e. CIN 2/3) firom low grade neoplasia (i.e. CIN 1) and 
healthy tissue. 

10 [0508] In Figure 73, step 130 uses the arbitrated spectra, BB and F, to determine four spectral 
masks -NEDspcc (no evidence of disease), Necrosisspec, [CE]spec (cervical edge/ vaginal wall), 
and [MU]spec (mucus/fluid). The focused, calibrated video data is used to determine nine image 
masks - Glarcyid, MucxiSyid, Bloodyid, Osyid, [ROI]vid (region of interest), [ST]vid (smoke tube), 
[SPjvid (speculum), [VW]vid (vaginal wall), and [FL]vid (fluid and foam). Step 1422 of Figure 73 

15 combines these masks to produce a hard ''indetenninate" mask, a soft '"indeterminate" mask, a 
mask identifying necrotic regions, and a mask identifying healthy (NED) regions. In the 
embodiment of Figure 73, steps 1424 and 1426 apply the necrotic mask and hard 
"mdeterminate" mask, respectively, prior to using the broadband spectral data in the statistical 
classifiers 134, while steps 1428 and 1430 apply the soft "indeterminate" mask and the NED 

20 mask after the statistical classification step 134. 

[0509] The embodhnent shown in Figure 73 can classify each interrogation point in step 1432 
as necrotic, CIN 2/3, NED, or Indeterminate. There may be some post-classification processing 
in step 1434, for example, for interrogation points having a valid fluorescence signal but having 
both broadband signals, BBl and BB2, eliminated by application of the arbitration metrics in 

25 step 128. The embodiment in Figure 73 then uses the final result to create a disease display 
overlay of a reference image of the tissue sample in step 138. Each of the masking and 
classification steps summarized above are discussed in more detail herein. 
[0510] In one altemative embodiment, the statistical classifiers in step 134 of Figure 73 
additionally include the use of fluorescence, image, and/or kinetic data. One altemative 

30 embodiment includes using different sets of spectral and/or image masks than those in Figure 73 . 
Also, one altemative embodiment includes using a different order of application of heuristic 
masks in relation to one or more statistical classifiers. In one altemative embodiment, kinetic 
data is determined by obtainii^ intensity data fcom a plurality of images captured during a tissue 
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scan, determining a relationship between corresponding areas of the images to reflect how they 
change with time, and segmenting the images based on the relationship. For example, an 
average kinetic whitening curve may be derived for tissue areas exhibiting similar whitening 
behavior. Whitening kinetics representative of a given area may be compared to reference 

S whitening kinetics indicative of known states of health, thereby indicating a state of health of the 
given area. In one alternative embodiment, the kinetic image-based data may be combined with 
spectral data to determine states of healtii of regions of a tissue sample. 
[0511] Figure 74 shows a block diagram 1438 depicting steps in the method of Figure 73 in 
further detail. The steps of Figure 74 are summarized below and are discussed in detail 

10 elsewhere herein. Steps 1440, 1442, 1444, and 1446 in Figure 74 depict determination of the 
spectral masks from the arbitrated broadband reflectance and fluorescence signals, as seen in 
step 130 of Figure 73. Steps 1448, 1450, 1452, 1454, 1456, 1458, 1460, 1462, and 1464 in 
Figure 74 depict determination of the image masks from the focused, calibrated video data, as 
seen in step 108 of Figure 73. The lines extending below tiiese mask determination steps in 

IS Figure 74 show how (in one embodiment) the masks are combined together, as indicated in step 
1422 of Figure 73. Steps 1466, 1468, 1470, 1472, 1474, 1476, 1478, and 1480 of Figure 74 
shows which masks are combined. Also important is the manner in which the masks are 
combined, disclosed in the detailed step explanations herein, 

[0512] The statistical classification step 134 from Figure 73 is shown in Figure 74 as steps 
20 1482, 1484, and 1486. Here, the pictured embodiment applies a necrosis mask 1424 and a hard 
"indeterminate" mask 1426 to the arbitrated broadband spectral data to eliminate the need to 
further process certain necrotic and indeterminate interrogation points in the classification step. 
Classification includes processing of broadband spectral data via wavelength region truncation, 
wavelength subsampling, and/or mean-centering. The processed data is then used in two 
25 different feature extraction methods. These include a principal component analysis (PCA) 
method used in the DASCO classifier step 1484 (Discriminant Analysis with Shrunken 
Covariances) and a feature coordinate extraction (FCE) method used in the DAFE classifier step 
1482 (Discrhninant Analysis Feature Extraction). Each of steps 1484 and 1482 extract a lower 
dunensional set of features from the spectral data that is then used in a Bayes' classifier to 
30 determine probabilities of classification in one or more tissue-class/state-of-health categories. 
The classification probabilities determined in stq)s 1482 and 1484 are combined in step 1486. 
Each of the classifiers in steps 1482 and 1484 are specified by a set of parameters that have been 
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determined by training on known reference data. One embodiment includes updating the 
classifier parameters as additional reference data becomes available. 

Spectral masking 

[0513] The invention comprises determining spectral masks. Spectral masks identify data 
5 from a patient scan that are potentially non-representative of regions of interest of the tissue 
sample. Spectral masks also identify data that are highly indicative of necrotic tissue or normal 
squamous (NED) tissue. In one embodiment, the spectral masks are combined as indicated in 
the block flow diagram 1438 of Figure 74, in order to account for the identification of spectrally- 
masked interrogation points in the tissue-class/state-of-health classification step 1432. Steps 
10 1440, 1442, 1444, and 1446 in Figure 74 depict the determination of spectral masks firom the 
arbitrated broadband reflectance and fluorescence spectra obtained during a patient scan and are 
discussed in more detail below. 

[0514] Step 1 440 in Figure 74 depicts the determination of an NEDspec (no evidence of 
disease) spectral mask using data fiom the fluorescence spectrum, F, and the broadband 

IS reflectance spectrum, BB, at each of the interrogation points of the scan pattern, following the 
arbitration and low-signal masking step 128. Applying the NEDspec mask reduces false positive 
diagnoses of CIN 2/3 resulting from the tissue-class/state-of-health classification step 134 in 
Figure 1 (and Figure 89). The NEDspec niask identifies tissue having optical properties distinctiy 
different fi:om those of CIN 2/3 tissue. More specifically, in one embodiment, the NEDspec mask 

20 uses differences between the fluorescence signals seen m normal squamous tissue and CIN 2/3 
tissue. These differences are not accounted for by tissue-class/state-of-health classifiers based on 
broadband reflectance data alone. For example, the NEDspec mask uses the collagen peak seen in 
the fluorescence spectra of normal squamous tissue at about 410 nm to distinguish normal 
squamous tissue from CIN 2/3 tissue. 

25 [0515] Figure 75 shows a scatter plot 1 500 depicting discrimination between regions of normal 
squamous tissue and CIN 2/3 tissue for a set of known reference data, according to one 
embodiment. Plotting fluorescence intensity at 460 lun (y-axis, 1502) against a ratio of 
fluorescence intensity, F(505 nm)/F(410 nm), (x-axis, 1504) provides good discrimination 
between regions known to be normal squamous tissue (blue points in Figure 75) and regions 

30 known to be CIN 2/3 tissue (red points in Figure 75). One component of the NEDspec 

discrimination metric is shown by line 1 506 in Figure 75, which divides a region of the plot that 
is predominately representative of normal squamous tissue (1508) from a region of the plot that 
is predominately representative of CIN 2/3 tissue (1510). The divider 1506 can be adjusted, for 
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example, to further reduce false positives or to allow detection of more true positives at the 
expense of increased false positives. 

[0516] In one embodiment, the fluorescence over reflectance ratio at about 430 nm is also 
mcluded in the NEDgpec metric to detemiine nomial columnar tissue sites that may not be 
5 identified by the component of the metric illustrated in Figure 75 (i.e. blue pomts on the right of 
line 1506). It is found that fluorescence of CIN 2/3 tissue at about 430 nm is lower relative to 
normal tissue, while CIN 2/3 reflectance at about 430 nm is higher relative to normal tissue, after 
application of a contrast agent such as acetic acid. 

[0517] Figure 76 shows a graph 1512 depicting as a function of wavelength 1514 the mean 
10 ^ broadband reflectance values 1 5 1 6 for a set of known normal squamous tissue regions 1518 and 
a set of known CIN 2/3 tissue regions 1520, used in one embodiment to determine an additional 
component of the NEDspec spectral mask. Figure 77 shows a graph 1522 depicting as a fimction 
of wavelength 1524 the mean fluorescence intensity values 1526 for tiie set of known squamous 
tissue regions 1528 and the set of known CIN 2/3 tissue regions 1530. The difference between 
15 curves 1528 and 1530 in Figure 77 is pronounced. Thus, a term is included in the NEDspec 
metric based on the best ratio of wavelengths found to maximize values of D in the 
discrimination equation. Equation 87, below: 

^ _ |//(F(;i)/i^(;iOL;^ -m{f{X)/f{a.%^ 

where \i indicates mean and o indicates standard deviation. Figure 78 shows a graph 1532 
20 depicting values of D in Equation 87 using a range of numerator wavelengths 1536 and 
denominator wavelengths 1538. According to flie graph 1532 in Figure 78, values of D are 
maximized using the fluorescence ratio F(450 nm)/F(566 nm). Alternately, other combmations 
of numerator wavelength and denominator wavelength may be chosen. 
[0518] A scatter plot depicting discrimination between regions of normal squamous tissue and 
25 CIN 2/3 tissue for a set of known reference data are produced by comparing the ratio F(450 
nm)/F(566 nm) to a threshold constant Then, a graph of trae positive ratio (TPR) versus false 
positive ratio (FPR) in the discrimination between regions of normal squamous tissue and CIN 
2/3 tissue are obtain^ using a tiireshold constant. For example, a TPR of 65% and an FPR of 
0.9% is obtained using a threshold constant of 4.5 1 . The ratio of false positives may be reduced 
30 by adjusting the threshold. 

[0519] Therefore, in one embodiment, the NEDspec mask combines the following three metrics: 

F(430)/BB(430)>Xi (88) 



wo 2004/095359 



PCTAJS2004/011820 



-132- 



F(450)/F(566)>X2 
F(460)>X3 • F(505)/F(410)-X4 



(89) 
(90) 



where xi, X2, X3, and X4 are constants chosen based on the desired aggressiveness of the metric. 
Equations 88-90 account for the distinguishing features of spectra obtained from regions of 

S normal squamous tissue versus spectra from CIN 2/3 tissue regions, as discussed above. 
[0520] Figures 79A-D illustrate adjustment of the components of the NEDspec mask metric 
shown in Equations 88, 89, and 90. Figure 79A depicts a reference image of cervical tissue 1 554 
from a patient scan in which spectral data is used in arbitration step 128, in NEDspec spectral 
masking, and in statistical classification of interrogation points of the tissue sample. Figure 79B 

10 is a representation (obgram) 1556 of the interrogation points (regions) of the tissue sample 
depicted in the reference image 1554 of Figure 79A and shows points that are "masked" 
following application of Equation 90. The obgram 1556 of Figure 79B shows that some 
additional interrogation points are masked as NED tissue by adjusting values of X3 and X4 in 
Equation 90 from {X3 - 120, X4 = 42} to {X3 = 1 15, X4 == 40}. Figure 79C shows interrogation 

15 points that are "masked" following application of Equation 89. The obgram 1570 of Figure 79C 
shows that a few additional points are masked as NED tissue by adjusting the value of X2 from 
4.0 to 4. 1 . Figure 79D shows interrogation points that are masked following application of 
Equation 88. The obgram 1584 of Figure 79D shows that a few additional points are masked as 
NED tissue by adjusting the value of Xi from 610 to 600. 

20 [0521] In one embodiment values of xi, X2, X3, and X4 in Equations 88, 89, and 90 are 

determined using multidimensional unconstrained nonlinear minimization. In one embodiment, 
the overall NEDsnec metric that results is as follows: . 



where the mean fluorescent intensity of normal squamous tissue is about 70 counts/iiJ at about 
450nm. 

[0522] Step 1442 in Figure 74 depicts the determination of Necrosisspec^ a necrotic tissue 
spectral mask, using data from the fluorescence spectrum, F, at each of the interrogation points 
30 of the scan pattern, following the arbitration and low-signal masking step 128. Unlike the other 
spectral masks (steps 1440, 1442, and 1446 in Figure 74), which are designed to reduce &lse 
positive diagnoses of CIN 2/3, tiie Necrosisspee mask identifies areas of necrotic tissue, thereby 
identifying patients with &irly advanced st^es of invasive carcinoma. 



25 



F(430)/BB(430)>600ct/^J OR 
F(450)/F(566)>4.1 OR 
F(460)>115 • F(505)/F(410)-40 
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[05231 In one embodiment, the Necrosisspec mask uses prominent features of the fluorescence 
spectra from a set of known necrotic regions to identify necrotic tissue. For example, m one 
embodiment, the NecrosiSspec mask uses the large porphyrin peaks of necrotic tissue at about 635 
nm and/or at about 695 nm in identifying necrotic tissue. Figure 80 shows a gmph 1598 
5 depicting fluorescence intensity 1600 as a function of wavelength 1602 from an interrogation 
point confirmed as invasive carcinoma by pathology and neax>tic tissue by impression, while 
Figure 81 shows a graph 1612 depicting broadband reflectance spectra BBl and BB2 for the 
same point. 

[0524] The graph 1598 of Figure 80 shows the distinctive porphyrin peaks at reference 
10 numbers 1604 and 1606. Concurrent with high porphyrin fluorescence at necrotic regions is a 
smaller peak at about 510 nm (label 1608), possibly due to flavin adenine dinucleotide (FAD), 
with an intensity greater than or equal to that of nicotinamide adenine dinucleotide (NADH) at 
about 450 tun (l^bel 1610). The FAD/NADH ratio is a measure of ischemia and/or hypoxia 
indicative of advanced stages of cancer. 
15 [0525] Thus, in one embodiment, the overall Necrosisgpec metric has one or more components 
indicative of FAD/NADH and one or more components indicative of porphyrin. In one 
embodiment^ the NecrosiSspec metric is as follows: 

F(510nm)/F(450nm)>1.0 AND 
F(635nm)/F(605nm)>1.3 AND 
20 F(635nm)/F(660nm)>1.3 AND 

F(635nm)>20ct/jJ 

where mean fluorescent intensity of normal squamous tissue is about 70 counts/^J at about 
450mn, and where the first Ime of the metric indicates FAD/NADH (FAD) and the remainder of 
the metric mdicates porphyrin. This metric requires all components to be satisfied in order for a 

25 region of tissue to be classified as necrotic. In one embodiment, the combination is needed to 
reduce false necrosis diagnoses in patients. The presence of porphyrin does not always indicate 
necrosis, and necrosis masking based solely on the detection of porphyrin may produce an 
xmacceptable number of false positives. For example, porphyrin may be present due to 
hemo^obin breakdown products following menses or due to systemic porphyrin resulting from 

30 medications, bacterial infection, or porphyria. Thus, the presence of both porphyrin and the 
indication of FAD must both be determined ui order for a region to be identified as necrotic by 
tiie NecrosiSspec metric in the embodiment described above. 
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[0526] Figure 82A depicts a reference image 1 61 8 of cervical tissue from the scan of a patient 
confirmed as having advanced invasive cancer, in which spectral data is used in arbitration step 
128, inNecrosisspcc spectral masking, and in statistical classification 134 of interrogation points 
of the tissue sample. Figure 82B is an obgram 1620 of the mterrogation points (regions) of the 

S tissue sample depicted in Figure 82A and shows points that are identified by application of the 
FAD component of the Necrosisspec metric above (1628), as well as points that are identified by 
application of the porphyrin component of the Necrosisspec metric above (1626). The overall 
NecrosiSspec mask above identifies points as necrotic only vdien both FAD and porphyrin are 
identified. In Figure 82B, interrogation pouits that are marked by both a blue dot (FAD 1626) 

10 and a green ring (porphyrin 1626) are identified as necrotic tissue by application of the 
Necrosisspec metric above. 

[0527] Step 1444 in Figure 74 depicts the determination of a cervical edge/vaginal wall 
spectral mask ([CEjspec) using data from the fluorescence spectrum, F, and the broadband 
reflectance spectrum, BB, of each interrogation point of a scan, following the arbitration and 

IS low-signal masking step 128. The [CE]spec mask identifies low-signal outliers conesponding to 
the cervical edge, os, and vaginal wall, which, in one embodiment, are regions outside an area of 
diagnostic interest for purposes of the tissue characterization system 100 of Figure 1. 
[0528] Figures 83, 84, 85, and 86 compare broadband reflectance and fluorescence spectra of 
cervical edge and vaginal wall regions to spectra of CIN 2/3 tissue. In one embodiment, these 

20 comparisons are used in a discrimination analysis to determine a [CEjspec spectral mask. Figure 
83 shows a graph 1638 depicting as a function of wavelength 1640 the mean broadband 
reflectance values 1642 for a set of knovm cervical edge regions 1644 and a set of known CIN 
2/3 tissue regions 1646. Figure 84 shows a graph 1648 depicting as a function of wavelength 
1650 the mean fluorescence intensity values 1652 for the set of known cervical edge regions 

25 1654 and the set of known CIN 2/3 tissue regions 1656, Figure 85 shows a graph 1658 depictmg 
as a function of wavelength 1 660 the mean broadband reflectance values 1 662 for a set of known 
vaginal wall regions 1664 and a set of known CIN 2/3 tissue regions 1666. Figure 86 shows a 
graph 1668 depicting as a function of wavelength 1670 the mean fluorescence intensity values 
1672 for the set of known vaginal wall regions 1674 and the set of known CIN 2/3 tissue regions 

30 1676. 

[0529] In one embodiment, features of the curves in Figures 83, 84, 85, and 86 are used in 
determining the [CEjspec spectral mask metric. For example, from Figures 84 and 86, it is seen 
that reflectance values for cervical edge/vaginal wall regions are lower than CIN 2/3 reflectance. 
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particularly at about 450 nm and at about 700 nm. From Figures 84 and 86, it is seen that there 
is a 'liump" in the fluorescence curves for cervical edge regions 1654 and vaginal wall regions 
1674 at about 400 nm, where there is no such hump in the CIN 2/3 curve (1656/1676). This 
causes the ratio of fluorescence intensity, F(530 nm)/F(410 nm), to be low at cervical 

5 edge/vaginal wall regions, relative to that of CIN 2/3 regions. From Figure 86, the mean 

fluorescence mtensity of vaginal wall regions 1674 is lower than that of CIN 2/3 regions at least 
from about 500 nm to about 540 nm. In one embodiment, these observations are combined to 
determine the overall [CE]spcc mask metric as follows: 

BB(450 nm) • BB(700 nm)/BB(540 nm) < 0.30 OR 

10 F^(530 nm)/F(410 nm) < 4.75, 

The top line of the metric above reflects the observation that the mean reflectance of cervical 
edge/vaginal wall tissue is comparable to that of CIN 2/3 tissue at about 540 nm and lower than 
that of CIN 2/3 tissue at about 450 nm and about 700 nm. The bottom line of the metric above 
reflects the observation that the fluorescence of a cervical edge/vaginal wall region may have a 

15 lower fluorescence at 530 nm than CIN 2/3 tissue and that the cervical edge/vaginal wall region 
may have a lower F(530 nm)/F(410 nm) ratio than CIN 2/3 tissue. 

[0530] Figure 87A depicts a reference image 1 678 of cervical tissue from a patient scan in 
which spectral data is used in arbitration and [CE]spec spectral masking. Figure 87B is an obgram 
1680 of the interrogation points (regions) of the tissue sample depicted in Figure 87A and shows, 
20 in yellow (1 684), the points that are "masked" by application of the [CE]spec metric above. 
White points (1682) in Figure 87B indicate regions that are filtered out by the arbitration and 
low-signal mask of step 128, while pink points (1686) indicate regions remaining after 
application of both the arbitrationAow-signal mask of step 128 as well as the [CE]spec spectral 
mask. 

25 [0531] Step 1446 in Figure 74 depicts the determmation of a fluids/mucus (|MU]spec) spectral 
mask using data fi-om the broadband reflectance spectrum, BB, at each interrogation point of the 
tissue sample following the arbitration and low-signal masking step 128. In one alternate 
embodiment, the fluorescence spectrum is used in place of or in addition to the broadband 
reflectance spectrum. The [MU]spec mask identifies tissue sites covered with thick, opaque, and 

30 light-colored mucus, as well as fluid that is pooling in the os or on top of the speculum during a 
patient scan. 

[0532] Figures 88, 89, 90, and 91 show steps in an exemplary discrimination analysis to 
determine a [MU]spec spectral mask. Figure 106 shows a gmph 1688 depictmg as a fimction of 
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wavelength 1690 the mean broadband reflectance values 1692 for a set of known pooling flxiids 
regions 1694 and a set of known CIN 2/3 tissue regions 1696, Figure 89 shows a graph 1697 
depicting as a function of wavelength 1698 the mean fluorescence intensity values 1700 for the 
set of known pooling fluids regions 1702 and the set of known CIN 2/3 tissue regions 1704. The 
S difference between curves 1694 and 1696 in Figure 88 is pronounced. Thus, in one embodiment, 
a term is included in the [MU]spec niask metric based on the best ratio of wavelengtti found to 
maximize values of D in the discrimination equation, Equation 91, as follows: 
^ \MiBB(X)/BB{A%,,„, -m{BB(X)/BB{Z'))^,\ 
,la'{BB{X)/BB{X'))^,,,^a'{BB{x)/BB{A,'))^ 

In one embodiment, values of D above are maximized using the broadband reflectance ratio 

10 BB(594 nm)/BB(610 nm). 

[0533] A scatter plot depicting discrimination between pooling fluids regions and CIN 2/3 
tissue regions for a set of known reference data are obtained by comparing tiie ratio of arbitrated 
broadband intensity, BB(594 nm)/BB(610 nm) to a threshold constant Then, a graph of true 
positive ratio (TPR) versus felse positive ratio (FPR) in the discrimination between pooling 

15 fluids regions and CIN 2/3 tissue regions are obtained using a threshold constant. For example, a 
TPR of 56.3% and an FPR of 0.9% is obtained using a threshold constant of 0.74. The ratio of 
false positives may be reduced by adjusting the threshold. 

[0534] Figure 90 shows a graph 1722 depicting as a function of wavelength 1724 the mean 
broadband reflectance values 1726 for a set of known mucus regions 1728 and a set of known 

20 CIN 2/3 tissue regions 1730. Figure 91 shows a graph 1732 depicting as a function of 
wavelength 1734 the mean fluorescence intensity values 1736 for the set of known mucus 
regions 1738 and the set of known CIN 2/3 tissue regions 1740. The difference between curves 
1728 and 1730 in Figure 90 is pronounced. Thus, in one embodiment, a temi is included in the 
[MU]spec metric based on the best ratio of wavelength found to maximize values of D in the 

25 discrimination equation, Equation 91 above. In one embodiment, this ratio is BB(456 
nm)/BB(542 nm). 

[05351 A scatter plot depicting discrimination between mucus regions and CIN 2/3 tissue 
regions for a set of known reference data may be obtained by comparing the ratio of arbitrated 
broadband intensity, BB(456 nm)/BB(542 nm) to a threshold constant. Then, a graph of true 
30 positive ratio (TPR) 1752 versus false positive ratio (FPR) 1754 in the discrimination between 
mucus regions and CIN 2/3 tissue regions are obtained using a threshold constant For example, 
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a TPR of 30.4% and an FPR of 0.8% is obtained using a threshold constant of 1 .06, The ratio of 
false positives may be reduced by adjusting the threshold. 

[0536] In one embodiment, the discrimination analysis illustrated in Figures 88, 89, 90, and 91 
lead to the overall [MU]spec mask metric as follows: 
5 BB(456mn)/BB(542nm)<L06 OR 

BB(594 mn)/BB(610 nm) > 0.74. 
The metric above combines the sites identified by the pooled fluids mask, as indicated by the 
bottom line of the metric above, with the sites identified by the mucus mask, as indicated by the 
top line of the metric above. 
10 [0537] Figure 92A depicts a reference image 1758 of cervical tissue &om a patient scan in 
which spectral data is used in arbitration and [MU]spec spectral masking. Figure 92B is an 
obgram 1770 of the interrogation points (regions) of the tissue sample depicted in Figure 92 A 
and shows, in yellow (1768), the points that are "masked" by application of the [MU]spec metric 
above. White points (1766) in Figure 92B indicate regions fliat are filtered out by the arbitration 
. IS and initial low-signal mask of step 128, while pmk points (1770) indicate regions remaining after 
application of both the arbitration/low-signal mask of step 128 as well as the [MU]spec spectral 
mask. 

Image masking 

[0538] The invention also comprises an image masking feature. Image masks identify data 
20 firom one or more images obtained during patient examination that are potentially non- 
representative of regions of interest of the tissue sample. Potentially non-representative data 
includes data that are affected by the presence of an obstruction, such as blood, mucus, a 
speculum, pooled fluid, or foam, for example. In one embodiment, a reference image of an in- 
situ cervical tissue sample is obtained jtist prior to a spectral scan, and image masks are 
25 determined firom the reference image to reveal where there may be an obstruction or other area 
that is not of diagnostic interest. Areas that are not of diagnostic interest include regions affected 
by glare, regions of the os, vaginal wall tissue, or regions that are otherwise outside the area of 
interest of the tissue sample. These areas may then be "masked" fi:om the analysis of spectral 
data obtained firom tissue regions that coincide with the obstruction, for example. The image 
30 masks are combined with each otiier and/or with the spectral masks, as shown in block 1422 of 
Figure 73 and as shown in Figure 74. The resultant masks include "hard" masks and "'soft" 
masks, described in more detail herein. Hard masks result in a characterization (or diagnosis) of 
'"Indetemiinate" at affected regions, while soft masking provides a means of weighting spectral 
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data according to the likelihood that the data is representative of clear, unobstructed tissue in a 
region of interest. 

[0539] In one embodiment, unage masks are combined and applied as indicated in the block 
digram 1438 of Figure 74, in order to account for the identification of image-masked 
5 interrogation points in the tissue-class/state-of-health classification step 1432. Steps 1448, 1450, 
1452, 1454, 1456, 1458, 1460, 1462, and 1464 in Figure 74 depict the detennmation of image 
masks firom the image data obtained around the time of the patient spectral scan- These image 
masks are discussed in more detail below. 

[0540] Figure 93 depicts image masks 1782, 1784, 1786 determmed firom a reference image of 
10 a tissue sample and conceptually shows how the image masks are combined with respect to each 
interrogation point (region) 1790 of the tissue sample, accordmg to one embodiment Generally, 
for a given interrogation point 1790 in tiie scan pattern 1788, tiie system deteraiines whetiier any 
of the features detected by the image masks, such as the os image mask 1784 and the blood 
image mask 1786, intersects that interrogation point (region) 1790. For certain image masks, a 
15 percent coverage is determmed for regions they mtersect For some unage masks, if any of the 
mask intersects a region, the region is flagged as * Wsked". 

[0541] In one embodiment, a backend process determines the coverage of one or more masks 
for each interrogation point of the scanning pattern. Given a known correspondence between 
image pixels and interrogation points, a given point is assigned a percentage coverage value for a 

20 feature determined by a given image mask, such as blood detected by die Bloodvid image mask 
1458 in Figure 74. The percentage coverage value corresponds to the number of pixels for the 
given interrogation point coinciding with the selected image mask feature, divided by the total 
number of pixels for that interrogation point. For example, if the blood mask for a given 
interrogation point coincides with 12 out of 283 pixels fliat cover the point, then the percentage 

25 coverage for that mterrogation point is 12/283, or 4.2%. 

[0542] Steps 1468, 1470, 1472, and 1474 m Figure 74 demonstrate how tiie image masks are 
combined m one embodunent, and steps 1466, 1476, 1424, 1478, 1480, 1424, 1426, 1428, and 
1430 in Figure 74 demonstrate how the combined masks are applied with respect to the tissue- 
class/state-of-health classifications at the spectral interrogation points, in one embodiment 

30 These steps are discussed in more detail herein. 

[0543] The image masks in Figure 74 are determined using image processing methods. These 
methods include color representation, spatial filt^ing, image thresholding, moiphological 
processing, histogram processing, and component labelmg methods, for example. 
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[0544] In one embodiment, images are obtained in 24-bit RGB fonnat. There are a number of 
ways to quantify image intensity and other image characteristics at each pixel. Most of the 
image masks in Figure 74 use values of luminance (grayscale intensity) at each pixel In one 
embodiment, luminance, Y, at a given pixel is defined as follows: 
5 Y = 0.299R + 0.587G + 0.114B (92) 

where Y is e}q)ressed in terms of red (R), green (G), and blue (B) intensities; and where R, G, 
and B range from 0 to 255 for a 24-bit RGB image. Some of the image masks in Figure 74 use 
one or more of the following quantities: 

redness^—*— (93) 

G-R G-B 

10 greenness = + (94) 

G+R G+B ^ ^ 

B-R B-G 

blueness = + (95) 

B+R B+G ^ ^ 

where R, G, and B are as defined above. 

[0545] Determination of the image masks in Figure 74 includes the use of one-dimensional (1 - 
D) and two-dimensional (2-D) filters. The types of filters used includes low-pass, smoothing 

IS filters and gradient, edge detection filters. The 1-D filters generally range in size from 3 to 21 
pbcels and the 2-D filters generally range from 3 x 3 to 1 5 x 35 pixels, although other filter sizes 
may be used. In one embodiment, box car filters are the preferred type of low-pass (smoothing) 
filters. Box car filters replace the value at the center of tiie filter support wifli an equally- 
weighted average of all pixels within the filter support. In one embodiment, the preferred types 

20 of gradient filters are Sobel and Laplacian of Gaussian filters. 

[0546] In one embodiment, the image masks in Figure 74 are determined using image 
thresholding, a subclass of image segmentation in which the image is divided into two segments. 
The criterion for assigning a pixel to one of the two segments is whether its value is less than, 
larger than, or equal to a prescribed threshold value. A binary unage may be obtained by 

25 marking pixels having values less than the threshold with zeros and the remaining pixels with 
ones. Some image masks are determined using multiple thresholding and/or dynamic 
thresholding, where the threshold for each pixel or group of pixels is computed dynamically 
fix)m image statistics, for example. 

[0547] In one embodiment, the determination of the image masks in Figure 74 mcludes binary 
30 morphological processing. Binary morphological processing is performed on a binarized 
(thresholded) unage to smooth object boundaries, change the size of objects, fill holes within 
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objects, remove small objects, and/or separate nearby objects. Morphological operators used 
herein include dilation, erosion, opening, and closing. An operator may be defined by (1) a 
binary mask or structuring element, (2) the mask origm, and (3) a mathematical operation that 
defines the value of the origin of the mask. In one embodiment, a 3 x 3 square structuring 
S element is used, and is generally preferred unless otherwise specified. 

[0548] In one embodunent, dilation mcreases tiie size of a binary object by half the size of the 
operator mask/structuring element. Erosion is the inverse of dilation and decreases the size of a 
bmary object For example, an erosion of a binary object is equivalent to the dilation of the 
background (non-objects). Opening is an erosion followed by a dilation, and closing is a dilation 
10 followed by an erosion. As used herein, dil(Img, n) denotes performing n dilation steps on 
image Img with a 3 x 3 square structuring element, and erod(Img, n) denotes performing n 
erosion steps on image Img with a 3 x 3 square structuring element. 

[0549] In one embodiment, the determination of the image masks in Figure 74 includes the use 
of histograms. Here, a histogram relates intervals of pixel luminance values (or other 
15 quantification) to the number of pbcels that fall within those intervals. In one embodiment, 
histogram processing includes smoothing a histogram using a 1-D low-pass filter, detecting one 
or more peaks and/or valleys (maxima and nmiima), and/or computing thresholds based on the 
peaks and/or valleys. 

[0550] In one embodiment, the determination of the image masks in Figure 74 includes 
20 component labeling. Component labeling is used to join neighboring pixels into connected 
regions that comprise the components (objects) in an image. Extracting and labeling of various 
disjoint and connected components (objects) in an image allows separate analysis for each 
object. 

[0551] In component labeling of a binary image using 8-connectivity, a connected components 
25 labeling operator scans the image by moving along the row until coming to a pixel p with a value 
V=l, then the operator examines the four neighbors of p that have already been encountered in 
the scan. For example, the four neighbors of p are (1) the pixel to the left of p, (2) the pixel 
directiy above p, and (3,4) the two pixels in the row above pixel p that are diagonal to pixel p. 
Based on this information, p is labeled as follows: 
30 • If all four neighbors have V=0, assign a new label to p, ELSE 

• If only one neighbor has V=0, assign its label to p, ELSE 

• If one or more neighbors have a value of 1, assign one of the labels 
to p and note the equivalences. 
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After completing the scan, the equivalent label pairs are sorted into equivalence classes and a 
unique label is assigned to each class. A second scan is made through the image, and each label 
is replaced by the label assigned to its equivalence class. Component labeling of a binary image 
with 4-connectivity may be performed shnilarly. 

5 [0552] In one embodiment, an image mask is determined using data from a representative 
image of a tissue sample obtained near to the time of a spectral scan of &e tissue (just before, 
during, and/or just after the spectral scan). In one embodiment, the representative image is 
obtained within about 30 seconds of the beginning or ending of the spectral scan; in another 
embodiment, the representative image is obtained within about 1 minute of the beginning or 

10 ending of the spectral scan; and in another embodiment, the representative image is obtained 
within about 2 minutes of the beginnmg or ending of the spectral scan. Other ranges of time in 
relation to the spectral scan are possible. In one embodhnent, there is only one reference image 
from which all the image masks are determined. 

Glare^d 

15 [0553] Step 1462 in Figure 74 depicts the detemiination of a glare mask, Glarcvid, for an image 
of a tissue sample. Glareyid indicates regions of glare in a tissue ims^e. Olarevid is also used in 
the computation of other image masks. Figure 94A depicts an exemplary image 1794 of cervical 
tissue used to determine a corresponding glare image mask, Glarcyid. Figure 94B represents a 
binary glare image mask, Glarcvid, 1796 corresponding to the tissue image 1794 in Figure 94A. 

20 [0554] The white specks of glare in the tissue image 1 794 in Figure 94 A are identified by the 
image mask 1796. The image mask is detemiined using an adaptive thresholding image 
processing procedure. Different thresholds are applied in different areas of the image, since the 
amount of illumination may vary over the image, and a threshold luminance indicative of glare 
in one area of the image may not indicate glare in another, lighter area of the image. In one 

25 embodiment, for example, an image of a tissue sample is divided into a 4 by 4 grid of equally- 
sized, non-overlapping blocks. A suitable glare threshold is computed for each block, and the 
subimage within that block is binarized with the computed threshold to yield a portion of the 
output glare segmentation mask, Glarcvid. Each block computation is independent, and blocks 
are serially processed xmtil the complete binary glare mask, Glarcyid, is completely calculated. 

30 For each block, multiple thresholds based on luminance value and/or histogram shape are 
computed and are used to detect and process bimodal distributions. 

[0555] Figure 95 is a block diagram depicting steps in a method of determining a glare image 
mask, Glarevid» for an image of cervical tissue. Step 1802 in Figure 95 indicates dividing an 



wo 2004/095359 



PCT/US2004/011820 



-142- 

image into a 4x4 grid of cells (blocks) 1804 and computing a histogram for each cell that is then 
used to determine thresholds 1806 applicable to that block. Each histogram correlates intervals 
of luminance values, Y, (Y ranging from 0 to 255) to the number of pixels in the cell (subimage) 
having luminance values within those intervals. 

S [0556] Step 1 806 in Figure 95 indicates detemmiing thresholds applicable to a given cell of the 
image. For example, Figure 96 shows a histogram 1842 for one cell of an exemplary image. 
Curve 1848 indicates a raw histogram plot for the cell (subimage), and curve 1850 indicates the 
curve after 1-D filtering using a 21-point box car filter. Quantities 1840 related to thresholding 
that are calculated from each histogram 1842 include Tpk (peak), Tvy (valley), Tip, Tg, Tdo, and 

10 T90, all of which are described below. The exemplary histogram 1 842 in Figure 96 shows bars 
mdicating values of Tpk (1852), Tvy (1854), T,p (1856), Ts (1858), Tdo (I860), and T90 (1862) for 
the cell histogram curve. The heavy dashed line (1854) indicates the final threshold chosen for 
the cell according to the method of Figure 95. 

[0557] The following describes the steps of the method 1 800 shown in Figure 95, according to 
IS one embodiment 

[0558] The method 1 800 in Figure 95 comprises calculating intended thresholds in step 1 806. 
Four thresholds are computed to decide whether the block (cell) contains glare: 

1 . Ts = mean + 3 * std vAigtq mean is the average intensity of the block and std 

its standard deviation. 

20 2. Tip = last peak of smoothed histogram. Smoothing is performed using a width 

5 maximum order statistic filter. 

3. Tdo = Lmax + 2 (Ldo - Lmax) where Lmax is the index (gray level) at 
which the 21-point boxcar filtered histogram, sHist, reaches it maximum value 
sHistMax, and Ldo is the first point after Lmax at which the filtered histogram 

25 value falls below 0.1 * sHistM ax. 

4. T90 is defined so that 90% of tiie graylevels greater than 21 0 are greater than 
T90. 

[0559] Next, the method 1 800 m Figure 95 mcludes a block (cell) glare detector in step 1810. 
The block (cell) glare detector assesses whether glare is present in the block and selects the next 
30 block if no glare is detected. The block is assumed to have no glare if the following condition is 
met: 

((Tip < Ts) AND (Ts < T90)) OR 
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((Tip < Tdo) AND (Tdo < T90)) OR 

((Tip < Tdo) AND (Tip < Ts) AND (Tip < T90)) OR 

((Tip < 0.8 * T90) AND (no valid glare mode as described in the bimodal 

histogram detection section belo^)). 
5 [0560] Next, the method 1 800 in Figure 95 con^rises selecting a candidate threshold, To, in 
step 1812. A candidate threshold Tc is chosen based upon the values of the intermediate 
thresholds Ts, Tip, Tdo and T90 according to the following rules: 

1. if(Tlp<T90): 

a if(Tdo<Tlp/2): 
10 i. if(Ts<Tlp): Tc= (Ts + Tlp)/2 

ii. else Tc = Tlp 
b. else Tc = min (Tdo, Tip) 

2. (Tip >=T90) High intensity glare . 

a. if(Ts<=T90): 

15 i. if((Ts<^100)AND(Tdo<=100)):Tc = max(Ts,Tdo) 

ii. else if ((Ts <=1 GO) and (Tdo > 100): Tc = mm (Tdo, Tip) 

iii. else Tc = min(Ts, Tdo) 

b. else 

i. if(Tdo<100):Tc = T90 

20 ii. else Tc = min (Tdo , T90). 

[0561] Next, the mediod 1 800 in Figure 95 includes detecting a bimodal histogram in step 
1 806. Step 1 806 detects bimodal histograms that are likely to segment glare from non-glare and 
uses the 21 point boxcar filtered histogram sHist to determioe Tvy after computing Tpk and 
Tcross, as described herein. To compute Tpk, sHist is searched backwards from the end until 

2S pomt Tpk where the value is greater than the mean and maximum of its 5 closest right and left 
neigbors and v\4iere Tpk is greater or equal to 10. Tcross is the point aft^ Tpk (in backwards 
search) wh^ the histogram value oosses over the value it has at Tpk. Iffbe histogram is 
unimodal, Tpk is equal to Lmax, the grayievel where sHist attains its max value, and Tcross is 
0. Tvy is the roinimum point on sHist between Tpk and Tcross if the following glare condition, 

30 called valid glare mode, is met: 

(Tpk > 175) AND (Tpk > Lmax) AND 
(sHist[tPk] < 0.6 * sHist[Lmax]) AND 
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((Tpk ^ Tcross > 20) OR (Tpk > T90)) AND 

((Tpk > (mean + (1.5 * std))) OR (Tpk > T90)). 

[0562] Next, the method 1 800 in Figure 95 includes selecting a final threshold in steps 1814, 
1816, 1818, 1820, 1822, 1824, and 1826. The final threshold selected depends on whether the 

5 histogram is bimodal or unimodal. For a bimodal histogram with a valid glare mode, the final 
threshold T is Tvy if 175 < Tvy < Tc. In all other cases (i.e. for unimodal histograms with a 
candidate threshold Tc and for bimodal histograms with a valley threshold Tvy ouside the range 
175 to Tc), Tc is chosen as the final threshold unless it can be incremented until sHistfrc] < 
0.01 * Shist[Lmax] or Tc > Tlim under the following two conditions. First, if a value L exists 

10 in the range [Tc,255] where sHist[L] > sHistlTc], define Lmin to be the gray value where sHist 
reaches its minimum in the range {TcJL]. Then, Tc should not be incremented beyond Lmin, 
and the Irniit threshold TLim = Lmin. If L< 150, then Tlim = 210. Secondly, ifL does not 
exist, Tlim = 210. 

15 [0563] Step 1448 in Figure 74 depic?ts the determmation of a general region-of-interest mask, 
[ROIlvid, for an image of a tissue sample. The general region-of-interest mask determines where 
there is tissue in an image, and removes ibG non-tissue background* [ROr|yid is also used in the 
computation of other image masks. Figure 97A depicts an exemplary image 1894 of cervical 
tissue used to detmnine a corresponding region-of-mterest mask, [ROrjvid, 1896 corresponding 

20 to the tissue image 1894 in Figure 97A. The mask 1896 excludes the non-tissue pixels in image 
1894. 

[0564] The [ROIlvid mask detects the general areas of the image indicative of tissue, and is 
determined by thresholding a pre-processed red chaimel image of the tissue and by performing 
additional processing steps to remove unwanted minor regions firom the thresholded image, 

25 explained in more detail below* 

[0565] Figure 98 is a block diagram 1900 depicting steps in a method of determining a region- 
of-interest image mask, [R0I]vid9 for an image of cervical tissue. The following describes the 
steps of the method shown in Figure 98 (1900), according to one embodiment. 
[0566] The method 1900 includes pre-processing in step 1902. First, smooth the red chaimel 

30 image by twice applying a 5x5 box car Gltcr. The filtered image is sRed. Next, compute a best 
dynamic threshold for sRed as follows. Create a foreground binary image of sRed using a 
threshold of 15. Create a glare mask binary image, glareMsk, using glare mask process Glarevid 
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above. Create a valid cervix pixel image, validPix, by binary AND-ing foreground and 
glareMsk inverse. Binary erode validPix, evalidPix = erod (validPix, 3). In evalidPix, find 
the top row containing the first valid pixel, topR; find the bottom row containing the last valid 
pixel, botR; the middle row is expressed as midR = (topR + botR)/2; then, set all evalidPix 
5 pixels above midR to 0. Compute mean, mean, and standard deviation, stdDev, of sRed on the 
region defined by evalidPix. The best dynamic threshold is then T = max(10, min (mean - 1 .5 * 
stdDev, 80)). Threshold sRed using T in step 1904. 

[0567] Next, the method 1900 in Figure 98 mcludes thresholding sRed usmg T in step 1904. 
Then, step 1906 is performing a binary component labeling using 4-way connectivity. Finally, 
10 step 1908 is computing the area of each object obtained in the previous step and selecting the 
largest object. Flood fill the background of the object selected in the previous step to fill holes. 
The result is the [ROIIvid mask. 

[0568] Step 14S0 in Figure 74 depicts the determination of a smoke tube mask, [ST]vid> for an 
IS image of a tissue sample. The smoke tube mask detennines whether the smoke tube portion of 
the speculxmi xised in tiie procedure is showing in the image of the tissue sample. The smoke 
tube mask also, identifies a portion of tissue lying over the smoke tube (which may also be 
referred to as "smoke tube" tissue) whose optical properties are thereby affected, possibly 
leading to erroneous tissue-class/state-of-health characterization. Figure 99A depicts an 
20 exemplary image 1932 of cervical tissue used to determine a corresponding smoke tube mask, 
[ST]vid, 1934 shown in Figure 99B. The smoke tube mask is determuied in part by isolating the 
two ^'prongs" holding the smoke tube tissue. The two prongs are visible in the image 1932 of 
Figure 99A at reference numbers 1930 and 1931. In some images, the prongs are not visible. 
However, the smoke tube tissue in these images (without visible prongs) is generally either a 
25 blue or blue-green color with ahnost no red component; and the smoke tube in these images is 
identified (and removed from consideration) by the general region-of-interest image mask, 
[ROrivid. 

[0569] Figure 100 is a block diagram 1938 depicting steps in a method of determining a smoke 
tube mask, [STjyid, for an image of cervical tissue. Image 1944 is an exemplary input image for 
30 which a corresponding smoke tube mask 1960 is computed. Image 1944 shows a ckcle 1945 
used in steps 1954 and 1956 of the method in Figure 100. 

[0570] The following describes the steps of the method shown in Figure 1 00, according to one 
embodiment. 
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[0571] The method 1938 m Figure 100 comprises step 1946, pre-processing the image. Pre- 
processing includes processing each RGB input channel with a 3x3 median filter followed by a 
3x3 boxcar filter to reduce noise. Step 1946 also includes calculating or retrieving the general 
ROI mask ROImsk ([ROI]vid, described above) and the glare mask glareMsk (Glarcvid, 

5 described above), and computing Ihe search image, srclmg, as follows. First, compute the 
redness image Rn. Set to zero all values in Rn that are oustide ROImsk. Autoscale the redness 
image to the [0,1] range. Then, compute srchimg, ^ch will be used at the final stages of the 
algorithm to compute a rough correlation to find the best circle location. Srchlmg is a linear 
combination of the redness and red images: srchlmg = (1 - A) * Rn + A * R. The linear 

10 weight factor A is in the range [0.2, 0.8]. Form validPix = ROImsk AND not(dil (glareMsk, 
3) . Compute mean, meanR, meanG, meanB of the RGB chaimels on the region defined by 
validPix. The weight A is initially computed as: A = max (0.5, min ((2 * meanR) / (meanG 
+ meanB), L5))- Remap tiie value of A mto the range [0.2, .8] , A = 0.2 + (0.6 * (A - 0.5)). 
Srchlmg is computed using the A factor deteraiined above. 

15 [0572] Next, the method 1938 in Figure 100 comprises a prong detector filter in step 1948. 
The prong detector is supplied to the red image, R and to an enhanced led image, RE to produce 2 
different prong images that will be arbitrated later. First, calculate the red-enhanced image, RE 
= R + max(R - G, R - B), Next, set up the prong detector filter. The filter is designed to be 
sensitive to smoke-tube prongs and to reject glare, edges and other features. The filter is a 

20 rectangular 35 by 15 separable filter. The horizontal filter His defined by H=[- 1.5 -1.5 -1.5- 
1.5-1.5 0 0 000 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 00000-1.5 -1.5 -L5 -1.5 -1.5]. The vertical 
filter V is a box car filter of length 15. Next, apply the prong filter to R and RE images yielding 
Rprong and Reprong. Clip filtered ims^es to 0 and autoscale to the range [0, 1]. Set the 
bottom half of each filtered image as well as die first 20 and the last 20 columns to 0 (tiiere are 

25 no prongs m these sections of images). Then, find a maximum value for each of the first 125 
rows of the 2 filtered images. Find the constant Rfact and REfact for each filtered image. 
These constants are defined as the mean of the maxima of the first 125 rows divided by mean of 
the first 125 rows. If (Rfact > Refact) use Rprong as the prong search image, iProng, 
otherwise use REprong. 

30 [0573] Next, the method 1 938 in Figure 1 00 comprises thresholding, component analysis, and 
prong selection in step 1950. Step 1950 is used to select prongs. First, threshold iProng image 
with a threshold of 0.2. Perform binary component labelling to obtain all regions (objects). 
Compute regions (objects) statistics, including area, centroid, and major and minor axis length. 
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Filter prong regions (objects). Discard each region (object) that statisfies any of the following 
criteria: 

1. Region size < 300. 

2. iProng maximum on object < 0.4. 

5 3. Region does not extend above row 100. 

4. Minor axis length >= 30. 

5. Region does not overlap with ROImsk. 

6. Region centroid is below row 140. 

7. Centroid y-value > 40 and object thinness (major axis length/minor axis 
10 length) <= 2. 

Choose as the main prong the brightest remaining region (i.e where the region maximum value 
is greater than tibe maxima from all other remaining regions). Filter all otiier prong regions 
based upon the distance from the main prong by calculating the distance from each region's 
centroid to tiie centroid of the main prong, and discarding the region if the intra-centroid 
15 distance > 160 or if Ihe intra-centroid distance <1 10. 

[0574] Next, method 1 938 in Figure 100 comprises validation of the selected prongs in step 
. 1952. For each retained prong object in step 1950, the following computations are pefonned to 
validate the selected prongs. Define pad, the rough distance from an object to its penmeter. 
Here, pad is set to 8. Form unages of the dimension of the bounding box of the object plus pad 

20 pixels on each side (OBJ). Crop the original object of the prong search image lOrig, from Ihe 
original unsmoothed red channel unage, Rorig, and form the binarized image BWProng. 
Compute internal region, intReg = erod (dil (OBJ, 2), 1). Compute object perimeter region, 
perObj = dil ((dil (OBJ, 2) AND not (OBJ)), 2). Compute mean and standard deviation, 
mean and std, of the object on the interior region, intReg, and the mean, pmean, on the 

25 perimeter region perObj. Compute left/right bias by computing locations of the center points of 
the object top row and bottom row, drawmg a line connecting those 2 points to divide the 
perimeter region, perObj, into 2 sections, calculating the mean value of iProng on each of the 
half perimeter sections, LperMean, RperMean, and using the means to compute lefr/rigfat 
biases, LRBias usmg LRBias = max (LperMean/RperMean, RperMean/LperMean). 

30 Discard any objects where the following holds: (mean/pmean < 1.4) OR (std > 0.55) OR 
((LRBias > 1.45) AND (mean/pmean < 1.48)). If more than 2 prong candidates are 
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remaining, keep the one closest to the main prong. If no prong candidates are left, the smoke 
tube mask is empty* 

[0575] Next, method 1938 in Figure 100 comprises template searching using circles m step 
1954. Step 1954 is used to qualify regions as smoke tube candidates. First, construct a binary 

5 mask, validCervix, of valid cervix pixel locations: by Computing or retrieving blood mask, 
bloodMsk, (Bloodvid, described below); Computing or retrieving glare mask, glareMsk, 
(Glarcvid, described above), then compute the bloodMsk using validCervix = ROImsk AND 
not(BWProng) AND not(dil (glareMsk, 3)) AND not(bloodMsk). Then, determine an x- 
coordinate value for the center, xCent, of the circle and radius, rad. For 2 prongs xCent is the 

10 half point between centroids of 2 prongs and rad is the half distance bewteen prongs + 5. For 1 
prong, choose a default rad of 85 and do a left-right search to decide wether the smoke tube is 
on the left or right The x-coordinate values, xCent, for each of the 2 search cux^les is the x- 
coordmate of flie prong centroid +A rad. The y-coordinate, yCent, is the y-coordinate of the 
prong centroid. For each circle center (xCent, yCent), find all points witioin rad that are in 

IS validCervix and compute the regional mean from the redness image. Then, find all points 
outside rad that are in validCervix and compute the regional mean from the redness image. 
Compute the contrast as the ratio of inner mean redness to outer mean redness and select the 
circle with minimum contrast Discard the previous circle if xCent is within rad /4 from the 
left or right edge of the image, since it cannot be a smoke tube. Then, use the search image, 

20 srchlnng, to perform an up-down search on the y-coordinate, yCent, to determine the actual 
smoke tube location usmg the x-coordinate xCent computed named above in section 2. Repeat 
the search with the redness image Rn if the results are unsatisfactory. A minimum and 
maximum value for yCent, yCentMIn and yCentMax are chosen as follows: 



1. 



yCentn/lin = -rad + yProngBot; where yProngBot is the mean of 
the bottom-most pomts of the prong(s), or the bottom-most point for a . 
single prong. 

For two prongs, yCentMax = yProngBot - (.75 ♦ rad) i.e. the cfrcle 
caimot extend beyond Ya rad below the bottom of the prongs. 

For one prong, yCentMax = min(yProngBot + rad /3, 150) i.e, the 
circle can go quite past the end of the prong, but not below the 150th 
row of the image. 



25 



2. 



3. 



30 
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Three more points spaced (yCentMax - yCentMm)/4 apart are computed between yCentMax 
and yCentn/lin. The search algorithm uses a total of yCent candidate points. For each yCent 
candidate, the inner/outer contrast for circles centered at (xCent, yCent) are computed using 
srchimg as follows: 

s 1 . Find all points within rad that are in validCervix and 

compute the regional mean from srchimg. 

2. Find all pomts outside rad that are in vaiidCervix and 
compute the regional mean from srchimg. 

3. Compute the contrast as the ratio of the inner mean value of 
10 srchimg to the outer mean value of srchimg and select 

the circle with mmimum contrast. 

Check to see that at least one of the 5 contrast numbers is less than 1 . If not, break out of the 
loop and proceed no further with this search. If at least one circle has contrast less than 1, 
choose the minimum and select a new set of five points centered around this one using the 
IS following steps: 

1 . If the top or bottom point was the minimxun, choose that 
point, tihe one below/above it, and three points evenly 

spaced in between them. 

2. If one of the three central points was the minimum, choose 
20 that point with the ones immediately below and above it, 

and two additional ones centered in the two spaces that 
divide those three. 

Using the new set of five points, go back to the computation of tiie iimer/outer contrast for 
circles using srchimg, discussed herein above, and proceed in this way until the distance 

25 between the five pomts is less than 3 pixels. When the distance between the points is less than 2 
pixels, exit the loop and choose the yCent with the current minimxmi contrast number as the 
final value of yCent for the circle. The contrast for the final circle must be less than 0.92 in 
order for the algorithm to find a valid cux^le. If that is not the case, then the search algorithm is 
repeated with the pure redness image, Rn instead of srchimg, which was a mixture of R and 

30 Rn. If the Rn search produces an acceptable result with contrast less tiian 0.92, then this value 
of yCent is used and we can proceed. Otherwise, there is no suitable circle and the 
segmentation mask will contain prongs but no circle. 
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f05761 Finally, method 1 938 in Figure 1 00 comprises producing the final smoke tube 
segmentation mask in step 1958. First, set the values of all pixels above the horizontal line 
inside the circle which is bisected by the center to 1 . This effectively casts a "shadow" straight 
upward fi:om the bottom of the image, and creates the effect that the smoke tube is coming 
5 straight down from outside of the image. The shadowed circle and prong images are combined 
to yield the final segmentation mask Clean up any stray non-prongs by performing a flood-fill 
of "on" valued regions with seeds in the first or thirtieth row of the image to select only objects 
that touch the first or thirtieth row of the image. 

Os^ 

10 [05771 Step 1460 in Figure 74 depicts the determination of an os image mask, Osyid, for an 
image of a tissue sample. The optical properties of the os region may differ from optical 
properties of the surroundmg tissue. In the method 1438 of Figure 74, the os image mask is used 
in soft masking to penalize data from interrogation points that mtersect or lie entirely within the 
OS region. Figures lOlA depicts an exemplary ims^e 1964 of cervical tissue used to determine a 

15 corresponding os image mask, Osvid, 1968, shown in Figure lOlB. 

[0578] The Osvid image mask is determined using a combination of thresholds from different 
color chaimels and using a binary component analysis scheme. An initial mask is formulated 
from a logical combination of masks computed from each color chaimel, R, G, B, and 
luminance, Y (equation 94). The four individual masks are computed using a thresholding 

20 method in which the threshold is set relative to the statistics of the colorplane values on the 

image region-of-interest (ROI). A component analysis scheme uses the initial mask to detect an 
OS candidate area (object), which is validated. 

[0579] Figure 1 02 is a block diagram 1 988 depicting steps in a method of determming an os 
mask, Osvid, for an image of cervical tissue. Image 1990 is an exemplary input image for which 
25 a corresponding os mask 2004 is computed. The following describes the steps of the method 
1988 shown in Figure 102, according to one embodiment. 

[0580] The method 1988 in Figure 102 includes image preprocessing in step 1992. 
Preprocessing includes computing luminance Y from RGB components Y = 0.299 * R + 0.587 * 
G + 0.1 14 B; smoothing RGB channels usmg 2 iterations of a 3x3 box car filter, and 
30 computing a ROI mask, ROImsk, ([ROr|vid) using the metiiod described herein above. Next, 
process the ROI mask by eroding ROImsk 14 times to obtain eROImsk = erod (ROImsk, 14). 
Compute annulus perimeter, annMsk: annMsk = dil ( (eROImsk AND not erod (eROImsk, 
. 1)), 4). This is a thick closed binary image which traces the edge of the ROI, usefiil in closing 
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the boundary around any os which might extend to the background Remove glare in ROImsk 
by logically AND-ing ROImsk with the complement of the glare mask (obtain as described 
above) to obtain centerROImsk. Then, compute a mean and standard deviation of each color 
channel (meanR, stdR, meanG, stdG, meanB, stdB, meanYi stdY) in ttie region 
5 specified by the centerROImsk. 

[0581] Next, the method 1988 in Figure 102 includes thresholding to produce an mitial 
segmentation mask in step 1994. First, cut-off centerROImsk around the annulus: 
centerROImsk = centerROImsk AND not (annMsk). Next, form a binary mask for each of 
the RGBY channels that represents pixels that exist in centerROImsk and that satisfy the 
10 following conditions: 

1 . mskR = (R pixels such that R < (meanR - .0.40* stdR)); 

2. mskG = (G pixels such that G < (meanG - .0-65 * stdG)); 

3. mskB = (B pixels such that B < (meanB - .0.75 * stdB)); 

4. mskY = (Y pixels such that Y < (meanY - .0.75 * stdY)). 
15 The resulting "initial" segmentation mask, msk, is then defined by; 

msk = centerROImsk AND mskR AND mskG AND mskB AND mskY. 

[0582] Next, the method 1988 in Figure 1 02 includes perforaiing a binary component analysis 
in step 1996. This step breaks up the segmentation mask into multiple objects. First, perform 
binary component labeling on segmentation msk. Remove all objects with size less than 125. 
20 Break apart all objects with size greater than 10000. For each object greater than 10000 
(thisObJMsk), do the following: 

1 . Compute mean value meanR and meanY for the area selected by thisObJMsk 
in the red and luminance channels. 

2. Set a new threshold for red and Y as follows: 
25 a. redT = 0.90* meanR 

b. lumT = meanY 

3. Break tiie object apart, or make it smaller to yield newObj, then complement 
thisObjMsk with the region that is not part of the newly broken-up region: 

newObj = thisObJMsk AND (R pixels such as R >= redT) AND (Y pixels 
30 such as Y >=lumT). 

thisObJMsk = thisOBjMsk AND (not(newObj). 
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4. Keep track of the original large image mask (thisObjMsk) that produces the 
smaller objects in step c. Create a large object mask IgObMsk for each 
thisObJMsk that is set to on for each large object which was foimd. 
[0583] Next, the method 1988 in Figure 102 includes performmg dilation, binary component 
analysis, and candidate selection in step 1998. Step 1998 is performed to find os candidates 
from the multiple binary objects produced m step 1996. Furst, dilate segMsk produced in step 
1996 twice to obtain bMsk « dil (segMsk, 2). Perform a component labeluig on bMsk. 
Discard objects of size less than 125 or greater than 23,000. For each remaining object, 
thisObjMsk, apply the following procedure to select candidates: 

1 . Compute mean, intMeanR, intMeanY, and standard deviation, intStdR, 
intStdY for red and luminance channel pixel values in thlsObjlMsk. 

2. Dilate thisObjMsk 7 times to yield dThisObjMsk = dil (thisObjMsk, 
7). 

3. Compute perimeter mask: 

a. thisObjPerim = dil ((thisObjMsk AND not(erod 
(dThisObjMsk,!))), 3). 

4. Compute mean, perMeanR, perMeanY, and standard deviation, 
perStdR, perStdY, for red and luminance channel pixel values in 
thisObjPerinn. 

5. Compute the following indicators: 

a. OS brightness (osBright)= intMeanY/ perMeanY. 

b. Perimeter uniformity (perUnif)'=perStdR/intStdR. 

6. An object is an os candidate if: 

((osBright < 0.85) AND (perUnif < L75)) OR 

((osBright < 0.7) AND (perUnif < 2.85) AND (part bf object came 
from large object as recorded in IgObjMsIc). 

[0584] Next, the method 1988 in Figure 102 includes performing candidate filtering and final 
selection in step 2000. The remaining os candidates are processed as follows. First, discard 
large non-os objects at the periphery of the cervix usmg the following algorithm: 
1 . Define a binary image with a centered circular area of radiiis 1 SO. 
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2. Discard the object if more than half of it is outside the circle and if 
perUnif > 0.9. This step is done by performing a logical AND of the 
object with the circular mask, counting pixels and comparing to the 
original size of object. 

If the number of remainmg objects is greater than 1, perform the following loop for each object: 

1 . Compute the centroid of the object, and compute the distance to the image 
center 

2. Exit if either: 

a. The distance to tiie center is less than 100 for all objects. 

b. No object lies wifliin 100 pixels of center and a single object 
remains. 

Discard the object with the highest perUnif, and go back to step b. Finally, step 2002 of the 
method 1988 in Figure 102 determines the final os mask by twice eroding the final mask 
obtained in step 2000. 

Bloodyj j 

[0585] Step 1458 in Figure 74 depicts the determination of a blood image mask, Bloodyid, for 
an image of a tissue sample. The presence of blood may adversely affect the optical properties 
of the underlying tissue. In the method of Figure 74, the blood image mask is used in soft 
masking to penalize data firom interrogation points that intersect or lie entirely within the blood 
regions. Figure 103A depicts an exemplary image 2008 of cervical tissue used to determine 
coirespondmg blood image mask, Bloodyid, 2012, shown in Figure 103B. 
[0586] In one embodiment, the Bloodvid image mask is similar to the Osvia image mask in that 
it is determined using an initial mask formulated from a logical combination of masks computed 
from each color channel R, G, B and lummance, Y. However, the initial Bloodyid image mask is 
formed as a logical "OR" (not "AND") combination of the four different masks, each designed 
to capture blood with different color characteristics. Blood may be almost entirely red, in which 
case the Red channel is nearly saturated and the green and blue channels are nearly zero. In 
other cases, blood is almost completely black and devoid of color. In still other cases, there is a 
mix of color where the red channel dominates over green and blue. In one embodiment, the 
Bloodyid mask identifies relatively large regions of blood, not in scattered isolated pixels that 
may be blood. The logical OR allows combination of regions of different color characteristics 
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into larger, more significant areas that represent blood. As with the Osvid mask, the Bloodvid 
mask is formulated by thresholding the initial mask and by performing component analysis. 
[0587] Figure 1 04 is a block diagram 2032 depicting steps in a method of detennining a blood 
image mask, Bloodvid, for an image of cervical tissue. The following describes the steps of the 
5 method 2032 shown in Figure 104, according to one embodiment 

[0588] The metiiod 2032 in Figure 104 includes image preprocessing in step 2034. 
Peprocessing includes computing luminance Y from RGB componaits Y = 0.299 * R + 0.587 * 
G + 0.1 14 ♦ B, and computing the ROI mask, ROImsk, ([ROI]vid) usmg the method described 
hereinabove. 

10 [0589] Next, the method 2032 in Figure 1 04 includes mask formation via thresholding in step 
2036. The following steps are used to produce an initial segmentation mask. First, four 
preliminary masks are generated to detect ''likely'' regions of blood, as follows: 

1 . To catch blood vMch is almost completely red, mskA 

mskA » ROImsk AND (B pixels such as B <15) AKD (G pixels such as 
15 G<15) AND (R pixels such as R > 2*max(G,B)). 

2. To catch areas where red dominates over green and blue, mskB: 
mskB = ROImsk AND (R pixels such as R > G * 3) AND (R pixels 
suchasR>B*3). 

3 . To catch really dark, almost black blood, mskC: 

20 mskC = ROImsk AND (R, G, B pixels such as R + G + B < 60). 

4. To catch dark, but not completely black blood, mskO: 

mskP = ROImsk AND (R, G, B pixels such as R + G + B < 150) 
AND (R pixels such as R <100) AND (R pixels such as R > max(G, B) * 
1.6). 

25 The final candidate segmentation mask, mskOrig, is computed as follows: mskOrig ^ 
mskA OR mskB OR mskC OR mskD. 

[0590] Next, the method 2032 in Figure 1 04 includes object selection using double 
thresholding in step 2040. The following steps are used to select regions that are blood candidate 
regions. First, a seed mask, seedMsk, is made by eroding mskOrIg twice. Then, to connect 
30 neighboring pixels, dilate mskOrig once, then erode the result once to obtain cIMskOrig. 
Finally, to eliminate spurious pixels and regions that are not connected to larger features. 



wo 2004/095359 



PCT/US2004/011820 



-155- 

compute mask, msk, by performing a flood fill of "on" valued regions of cIMskOrig with seeds 
in seedMsk. 

[0591] Next, the method 2032 in Figure 104 includes binary component analysis and object 
filtering in step 2042. Binary component labeling is performed on msk to select blood regions. 
For each labeled object the following steps are performed: 

1 .The Object mask is set to 0. Upon validation, the object mask is turned ON. 

2. An interior object is found by shrinking it once (1 erosion step) unless it 

disappears, in which case the algorithm reverts to the origmal object prior to 
erosion. 

3. Dilate tihie object OBJ 5 times, compute its perimeter and dilate the perimeter 5 

times: 

ObjPer = dil ((OBJ AND not(erod (dil (OBJ,5), 1))), 3). 

4. For both the interior and perimeter objects, the mean and standard deviation is 

found for the Red, Green, and Blue color-planes within the objects. The 
interior and perimeter mean luminance is found as the avemge of the Red, 
Green and Blue means. 

5. TW0 indicators are calculated which will help in the decision step: 

a. DarkBloodlndicator = (Perimeter Red mean) / (Interior Red mean). 
This mmiber is high for dark or black blood because there is more red in 
the perimeter than in the interior. 

b. BrightBloodlndicator = 

((Perimeter Green Mean + Perimeter Blue Bean) / Perimeter Red Mean) / 
((Interior Green Mean + Interior Blue Bean) / Interior Red Mean) . 
This number is large when the interior region has a much higher red 
content than green and blue as compared to the perimeter. 

6.1f the following three conditions are met, the region is considered to be a 
"noisy" feature which is most likely near the edge of the cervix. This 
determination affects the decision rules to follow: 

a. Interior mean Red < 40 
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b. (Interior standard deviation of Red > Interior mean Red) OR 
(Interior standard deviation of Green > Interior mean Green) OR 
(Interior standard deviation of Blue > Interior mean Blue) 

c. DarkBloodlndicator<5. 

7.The decision rules: If any of the following three rules are satisfied, then this 
object is Blood. Otherwise it is not. 

a. DarkBloocilndicator>2.S AND not'*noisy"; 

b. BrightBloodlndlcator > 2.25 AND not "noisy"; 

c. BrightBloodlndicator > 2.25 AND DarkBloodlndicator > 2.5 (in 

this case it doesn't matter if it's a "noisy"). 

8.1f the object is blood, it is turned ON in the final segmentation mask. 

[0592] Finally, the method 2032 in Figure 104 includes determinmg the final blood mask in 
step 2044. Step 2044 includes performing a flood-fill of all objects in which the seed objects 
were found to be blood. This yields the final blood segmentation . 

MuCUSvid 

[0593] Step 1464 in Figure 74 depicts the determination of a mucus image mask, Mucusvid, for 
an image of a tissue sample. The presence of mucus may affect the optical properties of the 
underlying tissue, possibly causing the tissue-class/state-of-health characterization in those 
regions to be erroneous. In the method 1438 of Figure 74, the mucus mask is used in soft 
masking to penalize data firom interrogation points that intersect or lie entirely within the mucus 
regions. Figure 105 A depicts an exemplary image 2064 of cervical tissue used to determine a 
corresponding mucus image mask, MucuSvid» 2068 shown in Figure lOSB. 
[0594] In one embodiment, the Mucusyid image mask is a modified blood image mask, tuned to 
search for greenish or bright bluish objects. Figure 106 is a block diagram 2072 depictmg steps 
in a method of determining a mucus mask, MucuSyid, for an image of cervical tissue. The 
following describes steps of the method 2072 shown in Figure 106, according to one 
embodiment. 

[0595] The method 2072 in Figure 106 includes preprocessmg in step 2074. Preprocessing 
includes processmg each RGB input channel with a 3x3 median filter followed by a 3x3 boxcar 
filter to reduce noise. Then, calculate or retrieve the following masks: 

1 . Glare mask (Glarewd): dilate glare mask once to yield glareMsk 
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2. ROImask([ROI]vid):ROImsk 

3 . Blood mask (Bloodvid): bloodMsk 

4. OS mask (Osvid): osMsk 

Compute a valid cervix pixels mask, validCervix, by AND-ing fhe ROImsk with the 
5 complement of the other masks as follows: validCervix = ROImsk AND not(glareMsk) 
AND not(bloodlVlsk) AND not(osMsk). 

[0596] Next, the method 2072 in Figure 1 06 includes mask formation via thresholding and ' 
morphological processing in step 2076. The following steps are used to produce an initial mucus 
segmentation mask. First, calculate the means, meanR, meanG and meanB, for the RGB 

10 channels on the validCervix region. Compute the difference, RGgap between the red and 
green mean: RGgap = meanR - meanG. Create a binary mask, mskOrig, according to the 
following rule: mskOrig = ROImsk AND (R,G,B pixels such as ((2*G - R - B) >= (10 - 
RGgap/3))). This rule selects regions where green is somewhat higher than either red or blue 
relative to the gap. Finally, process the binary mask with an opening morphological operator to 

IS obtain opMsk, as follows: 

1 . Perform two erosions with a 3-by-3 disk structuring element 

2. Perform one dilation with a 3-by-3 square structuring element 

3. Perform one dilation with a 3-by-3 disk structuring element 

[0597] Next, the method 2072 in Figure 1 06 mcludes object selection using double 
20 thresholding in step 2080. The foUowings steps are used to select objects from the initial 

segmentation mask by computation of seed points. First, a seed image, seed Msk, is computed 
by eroding opMsk 3 times. Then, opMsk is dilated twice then eroded once. Objects in 
opMsk are selected using seedMsk. For example, object I is selected at points where opMsk 
and seedMsk intersect, then selARsk is defined as the resulting object selection mask. 

25 [0598] Then, the method 2072 in Figure 106 includes binary component analysis and object 
filtering in step 2082. The following steps are applied to all objects selected in step 2080: 
1. Perform binary component labelling on all selected objects in selMsk. 

2.Set final segmentation mask to all O's. 

3. Compute area for each object in selMsk and discard any object with an area 
30 less than 1 000 pixels, update selMsk by removing discarded objects 
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4.Process all remaining objects in selMsk as follows (steps 2084, 2086): 

a. Compute mean and standard deviations of the red, green and blue 
smoothed images, meanR, meanG, meanBi stdR, stdG, stdB, for 
each object 

b. Compute the object perimeter for each object: 

i. Binary object, binObj, is dilated 15 times dilBinObj » 
dil(bmObj, 15). 

ii. Object perimeter is computed and then dilated: 

perBinObj = dil((dilBmObj AND not(erod (dilBinObj, 1)),4). 

c. Compute mean and standard deviations on each color channel, pmeanR, 
pmeanG, pmeanB, pstdR, pstdG, pstdB for each region's 
perimeter. 

d. Compute six decision rule indicators: 

i. Mucus Indicator 1: muclndl = (meanG/pmeanG) * 
(pmeanR/meanR) 

ii. Mucus Indicator 2: 

muclnd2 = (meanG/pmeanG) * (pmeanR/meanR) * 
(meanB/pmeanB) 

iii. Green bright indicator; gBrightInd = 3 * meanG - meanR - 
meanB 

iv. Local variation quotient: 

locVarQuo (stdR 4«tdG +stdB)/ (psdfR +p$tdG + pstdB) 

V. Target laser Indicator: 

targLasInd = (meanG * (pmeanR + pmeanB))/(pmeanG * . 
(meanR + meanB)) 

vi. Blue not too bright indicator: bNotBrightInd 

if ((meanB > meanR) AND (meanB > meanG)) 

bNotBrightInd = (meanG - meanR)/(2 * abs(meanB - 
meanG) 

else 
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bNotBrlghtlnd = 10. 

e. Object is not mucus object if the following holds: 

(muclndl < 1.25) OR (muclncl2 < 1.5) OR (gBrightInd < 100) OR 
(bNotBrighlnd<l)OR 

(targLasInd > 1.5) OR (locVarQuo > 1.75), 

f. If the object is selected as a mucus object, it is added to the final mucus mask. 

[0599] Step 1452 in Figure 74 depicts the detemunation of a speculum image mask, [SPJvid, 
for an image of a tissue sample. [SPjvid is used in hard masking in the tissue characterization 
method 1438 of Figure 74. Here, data from the interrogation points that intersect the speculum 
are removed from consideration in the tissue-class/state-of-health classification steps. Figure 
107A depicts an exemplary image, 2098, of cervical tissue used to determine the corresponding 
speculum image mask, [SPjvid, 2100, shown in Figure 107B. 

[0600] In one embodhnent, the speculum image mask is determined by finding circles near the 
bottom of the image. Projections of a number of different types of speculums resemble circles of 
different radii. In one embodiment, two types of circle searches are used: an outer bottom 
search and an inner bottom search. The outer bottom search finds points near the bottom edge of 
the general region-of-interest and infers circles from these points. If multiple circles result, they 
are evaluated to find the one that best models the curvature at the bottom of the region-of- 
interest. A circle that models this curvature well enough is used to form the speculimi 
segmentation mask, [SPjvid. 

[0601] If the outer bottom search does not produce a circle that models the ROI curvature well 
enough, then another search is performed to find a circle that models the curvature of a speculum 
within the ROL This is the inner bottom search, and may be necessary where there is significant 
reflection of light from the speculum. In the inner bottom search, a set of angular projections is 
formed based on a best guess of the center of curvature from the outer circle search. The 
projections are then analyzed to fmd a significant intensity trough near the end of the projections 
that agrees with the general expected location of a speculum at the bottom of the image. The 
projection analysis provides new points with which to model ckcles, and the resulting circles are 
evaluated using the image data to detect the presence of a speculum. 



wo 2004/095359 



PCT/US2004/011820 



-160- 

[0602] Figure 108 is a block diagram 21 12 depicting steps in a method of determining a 
speculum image mask, [SP]vid, for an image of cervical tissue. The following describes the steps 
of the method 21 12 shown in Figure 108, according to one embodiment. 
[06031 The method 21 12 in Figure 108 includes image preprocessing in steps 21 14 and 2116. 
The following steps are used to preprocess the image used in speculum mask computation. First, 
remove glare from the RGB image by performing the following: 

1 . Calculate or retrieve glare mask, glareMsk (Glaievid). 

2. Dilate glareMsk 4 times to obtain dilGlareMsk. 

3 . Filter the RGB values using dilGlareMsk to perform run-length 
boundary interpolation as follows: 

a. Raster scan each row ofdilGlareMsk to find all beginnings and 
ends of pixel runs. 

b. For each pixel P(x,y) in a given run specified by beginning point 
P(xb, y) and end point P(xe,y) in the intensity image, replace 
P(x,y) by half the linearly interpolated value at P(x,y) from P(xb,y) 
and P(xe,y), 

c. Raster scan each column of dilGlareMsk to find all beginnings 
and ends of pixel runs. 

d. For each pixel P(x,y) in a given run specified by beginning point 
P(x, yb) and end point P(x,ye) in the intensity image, add to P(x,y) 
half tiie linearly interpolated value at P(x,y) from P(x,yb) and 
P(x,ye). 

Then, smooth the RGB channels by filtering twice with a 5x5 box car filter. Finally, calculate or 
retrieve the ROI mask, ROImsk ([ROI]vid). Next, the method 21 12 in Figure 108 includes outer 
bottom ckcle detection in step 2120. The outer bottom circle detection is designed to find the 
best circular segmentation matching the bottom of ROimsk. Step 2120 mcludes flie following: 

1. Where width specifies the image width, compute the x4ocation of 7 columns 
(defined by none flie mtervals Q = i • width/10, where i = 1 to 9)- The x- 
locations are used to determine y-values. The resultant (x,y) pairs are used to 
find different candidate circles. 
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2,Four candidate circles - narrow, wide, left, and right - are calculated from the 
X values using the following matrix: 

a. Narrow circle: C3C5C7 

b. Wide circle: C2C5C8 

c. Left circle: C2C4C6 
d Right circle: C4C6C8 

3 .The y-values are determined by scanning the y -^is, at a givm x-position, 
starting at the bottom, until an "on" pixel is encountered in ROImsk. The 
same process is performed for 5 adjacent pixels to the right and left of the 
given x-positioiL The resultmg 1 1 y-values are averaged to obtain the y-value 
used for calculating circles at the given x-position, 

4. For each set of x values defined by the rows in the matrix above, the y values 

are computed as described above, and the resulting three pairs of coordinates 
are used to determine a unique circle intersecting these 3 points. 

5. A candidate circle is retained if: 

a. Radius R >250AND 

b. R<700AND 

c. The circle's center lies at a y value less than 240 (half the image height). 

[0604] Next, the method 21 12 in Figure 108 includes validation of the outer circle in step 
2122. The following steps are used to validate the outer ckcle: 

1 .If circles remain after the previous pruning, perform the following evaluation 
procedure: 

a. Compute candidate circle coater, draw perimeter at given radius and 
construct 2 offset regions from the drawn perimeter. 

b. The average intensity values, meanTop and meanBot, are calculated 
for each region on the red image. 

c. The BotTopRatio is calculated as the ratio of meanTop to meanBot 

L The top region is centered 1 0 pixels above the perimeter of the 
circle, and is 7 pixels in height For example, for a given (xO,yO) 
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point on the perimeter, the vertical region at xO comprises the 
pixels in the range (xO, yO+10) to (xO, yO+10-7). 
ii. Similarly, the bottom region is centered 1 0 pixels below the 
perimeter of the circle, and is 7 pixels in height 

d. The average int^ity values, meanTop and meanBot, are calculated 
for each region on the red image. 

e. The BotTopRatio is calculated as the ratio of meanTop to meanBot. 

2.The circle with the best fit to the actual speculum should niiniinize this ratio. If 
there is more than one circle remaining, the circle with minimum 
BotTopRatio is chosen. 

3.1f BotTopRatio > 0.55, the circle is rejected, and it is concluded that the outer 
bottom circle detection found no valid circle. 

If BotTopRatio < 0.55, the circle is kept as the initial result for the speculum segmentatioiL If 
the outer circle detection produces a circle with a strong enough representation of the speculum, 
then this is taken as the result and an iimer speculum search is not done. Otherwise the iimer 
speculum search is done. If no chrcle is found using the outer algorithm, perform the inner 
bottom speculum search. If the outer search finds a cncle, look at the BotTopRatio to 
determine whether it qualifies: 

1 . If BotTopRatio < 0,275, take the outer circle as the final segmentation 
mask and stop. 

2. If BotTopRatio >= 0.275, try the inner speculum search to see if it 
yields a satisfactory result 

[0605] Next, the method 21 12 in Figure 108 includes iimer bottom circle detection in step 
2126. The Inner bottom circle detection algorithm looks for ckcles within the ROI mask by 
calculating angular projections and looking for "valleys" in the projections to determine points 
that can be used to mfer circles. The resulting circles are evaluated with a scheme similar to the 
one for outer bottom circle detectioiL Step 2126 includes the following: 
1 .Angular projection center point selection: 

a. If an outer circle was detected, use the center point of liie outer circle. 

b. Else, use the pouit (n/2,1), where n is the width of the image. 
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2.The inner speculum search is done on the red color-plane R and a redness- 
cmhanced red image ERn. The search results fcom fbe two images R and 
ERn are evaluated as a group and the best result is taken from the entire set 
The redness enhanced red image is given by ERn «= (2 * R + Rn)/3, where 
Rn is the redness image defined in Equation 95. If no inner speculum is 
found from the redness enhanced red image, then the inner speculum search 
has determined that there is no identifiable inner speculum. The inner 
speculum search algoritiun is described in flie subsequent steps. 

S.Calculate angular projections as follows: 

a. Five x-values give the center of each projection as it crosses the bottom 
rowoftheunage: [CO C2 C4 C6 C8]. 

b. From these x-values, the angle thetaCtr, the central angle for tiie 
projection, is computed. 

c. For each angle thetaCtr, a projection sweeping out lb degrees (5 degrees 
on each side of thetaCtr) is calculated. 

d. For each 10 degree span, 50 equidistant line profiles (10/50 degrees) are 
used to calculate the projection. The profiles extend from the centra point 
to the point where the line at each angle crosses the bottom row of the 
image. 

e. The 50 profiles are averaged to yield the projection for each of the angles 
thetaCtr. 

f. Each projection profile is filtered with a 1 5 sample long boxcar movmg 
window average:. 

4.Each projection is searched backward to find the first "peak" in the projection, 
then search backwards again until the valley beyond that peak is found. This 
valley usually occurs near the boimdary between the speculum and the cervix. 
Not every projection will yield a good valley point V. The criteria for finding 
the valley V of a projection P are as follows; 

a. P (V) <= mean(P (V + k) for all k m [1 :12] (12 samples after V); 

b. P (V) <= mean(P (V + k) for aU k in [-12:-1](12 samples before V); 

c. P (V) <= P (V +k) for aU k in [-12:12]; 
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d. P (V) < P (V +k)-4 for some k in [V:lenglli(P)] (peak-valley is >= 4); 

e. For valley V, find the y coordinate value yv and check that yv > 300. 
5.Afler V is located, search backwards to find the point VMin where the first 

derivative of the projection is less than K * minSlope, where minSlope is 
5 the minimum slope between the valley V and the maximum of P(n) for n in 

[1 :V], and K is a constant parameter set to 0.3. VMin becomes the final point 
used for inferring circles from Ms projection. 

6.1f the number of points to infer circles (calculated firom the valleys as described 
above) is greater than 3, then as many circles as possible can be identified 
10 fi-om these points and evaluated. The drcles are chosrafinm the following 

matrix: 





CirclelDX - [ 13 5; 
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x-x-x 




234; 
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-XXX- 




123; 
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XXX-- 
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345; 
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1 24; 
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-x-xx 




1 34; 
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x-xx- 




235; 
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-xx-x 


20 


125; 


% 


XX--X 




1451; 


% 


X--XX 



where the elements of the matrix correspond to flie five projections computed 
above. If a specific projection j fails to yield an acceptable valley point, then 
all rows of the CirclelDX matrix which contain j are removed. 

25 7.A11 remaming rows m CirclelDX are used to select projections for inferring 

circles. The circles are calculated by first getting (x, y) coordmates for the 3 
points defined in ihe steps above, using the center of projection and the radius 
along the projection. A unique circle is fitted through the 3 points, unless 
pomts are coUinear, and circle center (xCent, yCent) and radius rad are 

30 computed. 
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[0606] Next, the method 21 12 in Figure 108 includes validation of the inner bottom circle in 
step 2128. The following steps are used to validate the inner bottom circle: 

LFor each circle, the circle is discarded if any of the following conditions applies: 

a. raci<250 (the circle is too small to be a speculum) 

b. yCent > (image height)/2 (centerof chrcle in lower half of hnage or beyond)- 
2.Each remaining circle is evaluated with the following technique: 

a A temporary image is defined for identifying three different regions 
specific to the circle. It is an 8-bit image with the followmg values: 

i. 1 for tihe "inner" region, which is the region between the circle and 
another circle whose center is 12 pixels below the original one. 

ii. 2 for the "bottom" region, which is a 12 pixel wide cncle drawn 
centered at 20 pixels below the original circle. 

iii. 3 for the *^op" region, which is a 12 pbcel wide circle drawn 
centered at 20 pixels above the original circle. 

iv. 0 for all other points in the image. 

b. Five sets of pixels are calculated on the temporary image. The average 
pixel value is calculated from the search image (Red or Redness enhanced 
Red) for each set of pixels: 

i. Top pixels, used to calculate AvgTop; 

ii. Bottom Pixels, used to calculate AvgBot; 

iii. Inner pixels, used to calculate Avgin; 

iv. Outer pixels (top and bottom), used to calculate AvgOut; 

V. Inner-bottom pixels (inner and bottom), used to calculate 
AvglnBot. 

c. Two ratios are calculated from these sets of pixels: 

V. lnOutRatio = Avgln/ AvgOut; 

vi. BotTopRatio = min([AvgBot / AvgTop, Avgln / AvgTop, 
AvglnBot /AvgTop]). 
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d. The InOutRatio gives an estimate of how closely the circle conforms to a 
low-intensity cervix-speculum boundary, and the BotTopRatio helps to 
evaluate how well the circle matches an intensity difference. 

e. To be a valid speculum representation, a circle should satisfy the 
5 following criterion: 

(InOutRatio < 0.70) OR (InOutRatio < 0.92 AND BotTopRatio < 
0.83 ). 

If no circles meet fliis criterion, then the algorithm detects NO inner 
speculum. 

10 f. The inner circle representmg the speculum is the circle from step e that 

has the minimum value of InOutRatio. 
g. If there is a resulting circle that has passed the validation procedure, 
evaluate to verify it is not a false positive by comparing the mean 
Imninance on two portions of the ROI, above the speculum and below the 

15 speculum. 

vii. Glare, blood and os are removed froin ROI to obtain dROI, where 

dROI = ROI AND not(glareMsk) AND not(blooclMsk) AND 
not(osM8k). 

viii. Compute mean luminance, meanLTop, on dROI region above 
20 circle. 

ix. Compute mean luminance, meanLBot, on dROI region below 
circle. 

X. If meanLBot > 0.8 * meanLTop and the bottom-most point on 
the mner circle is less than % of the image height, then the 

25 candidate is a false positive and is discarded. 

f0607] Fmally, the metiiod 21 12 in Figure 108 mcludes final determination of the specular 
segmentation mask in step 2128, The final segmentation mask is computed from the results of 
tiie iimer and outer speculum searches. If the outer search produces a satisfactory result and no 
inner search is done, the final mask is the one computed by the outer speculum search. If the 

30 outer search produces a satisfactory result and an iimer search is performed which also produces 
a result, the final segmentation mask is the logical OR of the inner and outer masks. If tiie outer 
search produces no result but the mner search produces a result, the final mask is the mask fi-om 



wo 2004/095359 



PCT/US2004/011820 



-167- 

the inner search. If neither search produces a result, the final segmentation is empty, indicating 
that the algorithm has determined that no speculum is present 

[0608] Step 1454 in Figure 74 depicts the determination of a vaginal wall unage mask, 
[VW]vid, for an image of a tissue sample. [VW]vid is used in hard-masking in the tissue 
characterization method 1438 of Figure 74. Figure 109A depicts an exemplary image 2190 of 
c^cal tissue used to determine the corresponding vaginal waU image mask, [VW]vid, 2194 
shown in Figure 109B. 

[0609] In one embodiment, the vaginal wall mask detects vaginal walls and cervical edges, 
including fomices and speculum blades. Here, the mask is determined using a filter shaped like 
a notch to emphasize the vaginal wall. This is similar to template matching in which the 
template is present along one dimension and the filter is constant along the other dimension. 
This achieves a projection-like averaging. 

[0610] After application of the filter in horizontal and vertical orientations, the resultant 
gradient images are thresholded and skeletonized. A heuristic graph searching method connects 
disconnected edges, and the edges are extended to the bounds of the image to form a full mask. 
Once the edges are extended, the edge lines are shadowed outward from the center of the image 
to form the final vaginal wall segmentation mask, [VW]vid. 

[0611] Figure 1 1 0 is a block diagram 22 1 8 depicting steps in a method of determining a 
vaginal wall image mask, [VWjvid, for an image of cervical tissue. The following describes the 
steps of the method 221 8 shown in Figure 110, according to one embodiment. 
[0612] Themethod2218 inFigure 110 includes preprocessing m step 2220. First, calculate or 
retrieve the glare, glareMsk, ROI, ROIMsk, and os, osMsk, segmentation masks. Calculate 
the luminance L firom the RGB sigpal using the formula: L = 0.299 * R + 0.587 * G + 0. 1 14 * 
B. Dilate glareMsk 4 tunes to obtam dilGlareMsk. Then, filter the RGB image using 
dilGlareMsk to perform run-length boundary interpolation as follows: 

1. Raster scan each row of dilGlareMsk to find all beginnings and ends of 
pixel runs. 

2. For each pixel P(x,y) in a given run specified by beginning point P(xb, y) 
and end point P(xe,y) in the mtensity image, replace P(x,y) by half the 
linearly interpolated value at P(x,y) fi:om P(xb,y) and P(xe,y). 

3. Raster scan each column of dilGlareMsk to find all beginnings and ends 
of pixel runs. 
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4. For each pixel P(x,y) in a given run specified by begimiingpom^ 

and end point P(x,ye) in the intensity image, add to P(x,y) half the linearly 
interpolated value at P(x,y) from P(x,yb) and P(x,ye). 

5 . Perform a 11x1 1 box car filter smoothing on dilGlareMsk regions only. 
5 Finally, smooth the filled RGB channels by filtering once with a 3x3 box car filter. 

> [0613] Next, the method 2218 in Figure 1 10 includes gradient image processing in steps 2222, 
and 2224 First, create a notch filter for detectmg the vaginal wall. The filter of length 22 is 
defined by the foUowing coefficients: [1 1 1 1 2/3 1/3 0 -1/3 -2/3 -1 -1 -1 -1 -2/3 -1/3 0 1/3 
2/3 111 1]. Then, normalize the filter: The average of the filter coefficients is subtracted from 

10 the filter in order to make it a zero-gain convolution kemel. Replicate rows 24 times to create a 
22 by 24 filter. Filter the limiinance image L with the vaginal wall notch filter to produce the 
vertical gradient image vGradlmg. Filter the luminance image witb the transpose of the notch 
filter to produce the horizontal gradient image hGradlmg. Clip gradient images to 0. Finally, 
perform die following thresholding and clean-iq) operations on each of the gradient images 

IS hGradlmg and vGradlmg: 

1 . Threshold the images at 975 to yield a binary object image. 

2. Perform a binary component labeling using 4-way connectivity. 

3. Compute regions statistics: area, centroid, major and minor axis length. 

4. Discard any object whose size is less than 1000 pixels. 

20 5. Discard any object which is within 80 pfacels of distance from the cento* of 

the image. 

6. Dynamically calculate the minimum allowable length, 
MinAllowedLength, for each object based upon the distance of its 
centroid (xCentroid, yCentroid) &om the center of the hnage (Cx, Cy) 

25 defined by Cx = (hnage width)/2 and Cy = (hnage height)/2. Let X be the 

distance of the centroid to the center of the image, x - sqrt ( (xCentroid 
- Cx)^ + (yCentrold - Cyf). 

MinAllowedLength scales the minimum allowed distance from 250 (at 
the image center) to 100 at the left or rightmost edge of the image and is 
30 defined by: 

MinAllowedLength = 250 - (15* x /25). 
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7. Discard any object >vifh a mjor axis length less than 
MinAllowedLength. 

8. Discard any object that is more than 50% outside of the image's ROL 

9. Discard any object that covers more than 5% of the OS. 

S [0614] Next, the method 2218 in Figure 110 includes skeletonization in step 2226. The binary 
images resulting from step 2224 are processed with a skeletonization algorithm that 
approximates the medial axis transform. The skeletonization algorithm works for either 
horizontal or vertical edges. For vertical edges, each row is scanned from left to right. Each 
time the pixel values transition from OFF to ON, the index of the ON pixel is remembered. If 

10 the first pixel in the row is ON, this qualifies as a transition. When there is a transition from ON 
back to OFF, the index of the last ON pixel is averaged with the index from the previous step to 
give the center pixel in the ON region. If an ON region extends to the last pixel in the row, then 
this last pixel is treated as a transition point All pixels between and including the first and last 
ON pixels are turned off except the center pixel. For horizontal edges, each column is scaimed 

15 from top to bottom. The same steps described hereinabove are repeated for the columns instead 
of the rows. 

[0615] Next, the method 2218 in Figure 110 includes edge linking and extension in steps 2226, 
and 2228. The skeletonizations are processed with a heuristic graph-searching method which 
connects slight gaps in the skeletonized images and extends tiie edges to the image boundary. 
20 The following images and parameters are used by the edge linkmg algorithm: 

• Horizontal and v^ical skeletonized edge ime^e, vSkellmg, hSkellmg 

• Input label matrix, LblMat This is found by labeling matrix output from the 

connected components analysis, where discarded regions have been removed 
from the label matrix by setting their pixel values back to 0. 
25 • Horizontal and vertical edge orientation, vEdgeOrient, hEdgeOrient. 

• Skeletonized input label matrix, skLblMat This is a copy of LblMat where all 

tiie pixels which are OFF in the skeletonized image are set to 0 in skLblMat 

• Gap = 16.0, the maximum allowable gap to fill in for a disconnected edge. 
The following are searching methods that are implemented. 

30 l.Search for Edge Pixels: For bofli the horizontal and vertical edge images, the 

images are raster searched to locate edges within them, 
a. The vertical edge image, vSkellmg, is searched by row raster scanning to 
ensure that the first point in an edge is encountered. 
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b. The horizontal edge image, hSkellmg, is searched by column raster 
scanning to ensure that the first point in an edge is encoxmtered. 

c. When a point is encountered, the algorithm references skLblMat to see if 
that point has a positive label, indicating that this edge has not yet been 
processed. If so, the edge connection and edge extension routines 
desoribed in the steps below are executed starting from this point 

2.£dge Connection. The edge connection routine starts from the point from 
which it is called. The routine keeps a list of the pomts encountered in the 
edge. The search is executed only for points witii the same label m 
dilGlareMsk. 

a. Create Label matrix SkLblMat as described above. 

b. Find second point: 

i. Startmg from the first point, do a search in a rectangular region of 
size 2*( Gap +1.5) + 1 centered about the first point 

ii. The second point will be the point which is ON in the edge image 
which is closest to the first point, and which is not already part of 
any other linked edge (must have same label value as the first 
point). 

iii. Fill m the gap between the first pomt and the second point The 
Gap filling algorithm is described below in step 3. 

iv. If this edge begins at a point "sufficiently close" (with respect to 
Gap) to another edge, set a flag to prevent extension of the 
beginning of this edge. 

V. If no second point is found, or if the second point is part of another 
edge which has afready been linked, erase this edge in the output 
edge image (see Edge Erasing description below) and in 
SkLblMat stop processing this edge, and continue the loop to 
look for the next edge. 

c. Find the third point 

i. Starting from the second point, do a search in a rectangular region 
of size 2*( Gap +1 .5) + 1 centered about the second point 
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ii, The point wiU be the point wUch is ON mtiie edge image 
which is closest to the second point, and which is not aheady part 
of this or any other linked edge (must have same label value as the 
first point). 

5 iii. Fill in die gap between the second point and the third point. 

iv. If no third point is found, or if the third point is part of another 
edge which has already been linked, erase this edge in the output 

edge image, stop processing this edge, and continue the loop to 
look for the next edge. 

10 d. After three points in tiiis edge are discovered, there is enough information 

to infer a search direction, and from here on out all searches m the Edge 
Connection are directional. Steps for computing the search location are 
listed below. 

e. Starting with the search for the fourth point, the following steps are 
15 iteratively performed until no ftirther pixels in this edge can be foxmd: 

i. the search dnection: North (N), South (S), East (E), West (W), 
NorfhEast (SE), NorthWest (NW), SouthEast (SE).or SouthWest 
(S W) is computed by the steps described below* 

ii. Check the edge length, ifit is greater than 2048, break out of the 
20 loop because this edge must have looped back upon itself. 

iii. Find the next point in the given search direction: If no further 
points were found, check to see if the edge length is less than 120. 

1 . If edge length < 120, erase edge and break out of this loop 
to continue the processmg to find other edges (back to step 

25 1). 

2. If edge length >= 120, keep edge end break out of loop and 
continue with step f). 

iv. Fill in the gap between the current point and the new point 

V. If the new point belongs to an edge vMch was akeady linked by 
30 this algorithm, do the following: 
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1. Ifthe current edge is less than 40 pkels in leng&,e^ 
edge. Break out of the loop and continue searching for 
forther edges (back to step 1). 

2. Otherwise, the edge vAW be kept, but a flag is set so that the 
end offhis edge is not extended. Break out of the loop and 
continue with step £ 

vi. increment the edge length so that the new point becomes the 
current point for the next iteration. 

vii. Continue with step i) to contmue processing. 

f. At this point, a valid edge has been detected. This edge will then be 
extended in the both directions to the boundary of the image unless either 
edge (or both) is flagged for not extending. The edge extension steps are 
described below in step 5. 

g. Check to see if an extension passed through the center of the ims^e 
(defined by a circle of radius 80 centered at the geometrical center of the 
image). 

i. If an extension did pass through the center of the image, erase this 
edge and all of its extensions. 

ii. Otherwise, relabel this edge in tibe Label matrix to have value -1, 
and draw the extensions on tiie output edge image, simultaneously 
labeling the corresponding pixels in the Label matrix with value 

3.Gap Filling method: 

a. Check to see if there is no gap, i.e. if the edge is akeady connected. 
Where (xl ,yl) and (x2,y2) are the new point and the current point, if 
abs(xl-x2)<2 and abs(yl-y2)<2, then there is no gap to fill, and the Gap 
Filling processing stops. 

b. Remove the ^'New pixel" firom the edge vectors so that it can be replaced 
with a set of filled-in pixels. 

c. Check for special cases where xl=x2 or yl=y2. In either of those two 
cases, the Gap Filling is accomplished by simply turning on every pixel 
which lies between the two pixels hi the output Edge image. 
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d. For the case where xl is not equal to x2 and yl not equal to y2,adiagonal 
line needs to be drawn to fill the gap. 

i. This is done first by computing an equation for the line wiiich 
connects the two pomts. 

ii. If the slope is greater than 1, iterate fi:om y=yl to y2, and compute 
the X value for each y value. For each (x,y) turn on the 
corresponding pixel in the output Edge image and in skLabMat 

iii. If the slope is less than 1 , iterate firom x=xl to x2, and compute the 
y value for each x value. For each (x,y) turn on the corresponding 
pixel in the output Edge image and in skLabMat 

e- Finally, all of the new pixels are added to the edge vectors in order bom 
the current pixel to the new one. The corresponding pixels in skLabMat 
are set to the label value -2. 

4.Computing Search Direction: 

a. Two pixel locations are used to infer a search direction. 

i. The first point is the geometric average ofthe two most current 
pixels in the edge. 

ii. Ifthere are less flian 6 pixels m the edge, the second point is the 
average of the first and second pixels in the edge. 

iii. If there are more than 6 pixels in the edge, the second point is the 
average ofthe fifth and sixth most current pixels in the edge. 

b. For the two pixels (xl,yl) and (x2,y2), the search direction is computed as 
follows: 

i. Compute the angle formed by the two points using the ATAN2 
fimction: 

angle = atan2(yl-yO,xl-xO) * 180 / n\ 

ii. If angle is in the mterval [-22.5, 22.5], the search direction is E. 

iii. If angle is m the interval [22.5, 67.5], the search direction is SE. 

iv. If angle is m the interval [67.5, 1 12.5], the search direction is S. 

V. If angle is in the interval [1 12.5, 157.5], the search direction is SW. 
vi. If angle is in the interval [-67.5, -22.5], the search direction is NE. 
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vii. If angle is in the interval [-1 12.5, -67.5], the search direction is N 

viii. If angle is in the interval [-157.5, -1 12.5], the search direction is E. 
ix. Othenvise, the search direction is W. 

5.Edge Extension: 

a. It is the default to extend both the beginning and end of the edge. 
However, during the edge connection steps, if it is discovered that the 
edge originates close to a different edge, the edge is connected to the 
different edge and is not extended. If an edge ends by mergiog with 
another edge, the end of the edge is not extended. 

b. For both the beginning and &e end of the edge: 

i. For Vertically oriented edge images (vEdgeOrient): 

1 . If the y^oordinate for the firstAast point of the edge is less 
than the image height/6 or greater than 5*height/6, extend 
the beginning/end of the edge usmg the local slope method 
(described below). 

2. Otherwise, extend the beginnmg/endofthe edge using the 
global slope method (described below). 

ii. For Horizontally oriented edge images (HEdgeOrient): 

1. If the x-coordinate for the first/last pomt of the edge is less 
than the image width/6 or greater than 5* width /6, extend 
the begmning/end of the edge using the local slope method 
(described below). 

2. Otherwise, extend the beginning/end of the edge using the 
global slope method (described below). 

c. Local Slope Extension: This method uses the slope of the edge near its 
beginning/end to determine the slope of the extending line. 

i. Compute two points for slope computation: 

1 . the average of the four pixels from the beginning/end of the 
edge; and 

2. the average ofthe 6th through 9th pixels from the 
beginmng/end of the edge. 
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ii. Usmg the two computed points, the edge is extended firo^ 
beginning/end point using a line of the computed slope until it 
reaches the edge of the ims^e. 

d. Global Slope Extension: this method uses pixel values between 20% and 
80% of the length along the edge to guess the "average" slope of this edge. 
Then the beginning/end of the edge is extended using this slope. 

i. If the edge has edgeLen pixels m it, select the points in the edge 
with the foUowmg indices: 

1. beglDX = round(edgeLen*0.2); pointA= 
edge(begiDX); 

2. endlDX = round(edgeLen*0.8); pointB = 
edge(endlDX). 

ii. Compute the slope using pointA and pointB, and use a line of 
this slope to extend from the beginning/end of this edge. 

e. After the extension is computed, the extended pixels are tumed ON in the 
output edge image, and the corresponding pixels in skLabMat are 
assigned value -2. 

6.Edge Erasing. 

When an edge is to be erased check to verify that for each pixel in the edge 
and its extension the label for the pixel is > 0. If so, set the value in the output 
Edge image and the label matrix to 0. This method assures that pixels in 
another edge that has akeady been linked are not erased (the two edges might 
have crossed). 

[0616] Fmally, the method 221 8 m Figure 1 1 0 mcludes mask computation m step 2230. The 
output of the Edge Linking algorithm is used to generate the vagmal wall mask in the following 

way: 

LVertical connected-edge image: VConnlmg, a cumulative sum, is calculated 
for each row, starting from the center and extending both to the left and to the 
right 

2.Hori2X)ntal connected-edge image: HConnlmg, a cumulative sum, is 
calculated for each column, startmg from the center and extending both 
upv^ard and downward. 
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3 .The two cumulative sums are thiesholded at >«=1 and OR-ed together to yield 
the final vaginal wall mask. 

[0617] Step 1454 in Figure 74 depicts the determination of a fluid-and-foam mask, [FLjvid, for 

5 an image of a tissue sample. This mask identifies fluid and foam regions appearing on tissue 
samples and is used in hard masking in the tissue characterization method 1438 of Figure 74. 
Figure 1 1 1 A depicts an exemplary image 2234 of cervical tissue used to determine the 
corresponding fluid-and-foam hnage mask, [FL]v5d, 2238 shown in Figure 1 1 IB. 
[06181 ^^'^ embodiment, the fluid-and-foam hnage mask identifies regions where excess 

10 fluids and/or foam collect on cervical tissue. Excess fluid or foam can collect near the speculum, 
around or in the os, and/or in the folds between the vaginal walls and the cervix, for example. 
One embodiment of the fluid-and-foam hnage mask, [FL]vid, uses a measure of whiteness and a 
measure of blue-greenness to identify regions of fluid/foam. After extracting white and blue- 
green color features, thresholdmg and validation is performed to produce the final fluid-and- 

15 foam image mask, [FL]vid- 

[0619] Figure 1 12 is a block diagram 2258 depicting steps m a method of determining a fluid- 
and-foam image mask, [FL]vid, for an image of cervical tissue. The following describes the steps 
of the method 2258 shown in Figure 1 12, according to one embodiment. 
[0620] The method 2258 m Figure 1 12 includes preprocessmg in step 2260. First, remove 

20 glare fi:om the RGB image. Retrieve or compute glare mask, glareMsk. Dilate glareMsk 4 
tunes to obtam dilGlareMsk. Next, retrieve or compute ROI mask, ROIMsk. Fmally, smooth 
each of the RGB chaimei using a 3x3 box car filter to remove noise. 
[0621] Next, the method 2258 m Figure 1 12 includes hnage color feature calculation in step 
2262. This step computes a *Svhiteness" hnage, Wimg, and a "gre^-blueness" image, GBImg. 

25 First, calculate the lummance L from the RGB signal usmg the formula: L = 0.299 ♦ R + 0.587 * 
G + 0.1 14 * B. Next, compute, normalize and threshold Wimg as follows: 

1. WImg = abs((R - G)/( R + G))+abs((R - B)/( R + B))+abs((G - B)/( G + 
B)). 

This operation is a pixel-wise operation and is performed on each 
30 pixel sequentially. 

2. Normalize WImg: WImg = 3 - WImg. 

3. Set low luminance pixels to 0 (low luminance pixels are unlikely to be m 
the fluid and foam regions): 
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If L < mean(L), WImg = 0, 
Finally, compute, normalize and threshold BGImg as follows: 

L BGImg ^ (abs((R +30- G) /( R +30+ G))+abs((R +30- B) /( R +30+ 
B))+abs((G-B)/(G + B))). 

This operation is a pixel-wise operation and is performed on each pixel 
sequentially. 

2. Normalize BGImg, BGImg = 3 "BGImg. 

3. Set low luminance pixels to OGowliiminance pixels are imlikely to be in 

tiie fluid and foam regions): ^ 
If L < 0.65 * mean(L), BGImg = 0. 

[0622] Next, the method 2258 in Figure 1 12 includes processing and segmentmg bright green- 
bluish legions in steps 2264, 2266, 2268, 2270, 2272, 2274, and 2276. These steps are 
performed as follows: 

LRetrieve or compute glare mask, giareMsk. 

2. Fill glare regions of BGImg using giareMsk to perform run-length boundary 

interpolation as follows: 

a. Raster scan each row of giareMsk to find all beginnings and endsof pixel 
runs. 

b. For each pixel P(x,y) in a given run specified by beginning point P(xb, y) 
and end point P(xe,y) in the intensity image, replace P(x,y) by half the 
linearly interpolated value at P(x,y) fsom P(xb,y) and P(xe,y). 

c. Raster scan each column of giareMsk to find all beginnings and ends of 
pixel runs. 

d. For each pixel P(x,y) in a given run specified by beginning point P(x, yb) 
and end point P(x,ye) in the intensity image, add to P(x,y) half the linearly 
interpolated value at P(x,y) firom P(x,yb) and P(x,ye), 

3 . Eliminate low intensity areas using a threshold of 1 .5: 

If BGImg < 1.5, BGImg =1.5. 

4 . Rescale the BGImg to [0, 1]: 

BGImg = BGImg - min(BGImg))/(3 - min(BGImg). 
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5. Compute thresholds from image statistics and perfoim thresholding. 

a. Compute image mean mtensity, Imean, for BGImg > 0. 

b. Compute image standard deviation of intensity, IstdDev, for BGImg > 0. 
Compute threshold thGB, thGB = Imean +1.63 * IstdDev. 

c. Apply threshold limits: 

if thGB < 0.80, thGB - 0.80; 
if thGB > 0.92, thGB » 0.92. 

d. Threshold to get the initial green-bluish fluid and foam mask GBMask 
ifBGImg> thGB, then 

GBMask ="1; 
else 

GBMask = 0. 

6. Perfonn motphological processing to fill small holes and smooth boundaries of 

the found regions in GBMask: 

a. Dilate the segmentation mask GBMask twice, GBMask dil(GBMask, 
2). 

b. Erode the resultant mask three times, GBMask = erode(GBMask, 3). 

c. Dilate the resultant mask once, GBMask = dil(GBMask, 1). 

7. Perform binary region labeling and small region removal: 

a. Perform a connected components labeling, described above, to label all 
found regions. 

b. Compute each region area, area, and eccentricity, ecc. 

c. Remove small and round regions and small line segments that are not 
likely to be the fluid and foam regions: 

If ((area < 1000) AND (ecc < 0.70)) OR ((area < 300) AND (ecc > 
0.70)) OR (area < 1000), remove region. 

S.Green-Bluish feature validation for each found region is based on the original 
RGB values: 
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a. For each found region, retrieve the mask, Imsk, and compute the mean 
intensities ^thin the region for each of the red, green and blue chaimels as 
MRed, MGreen and Mblue. 

b. If the found region is tissue-like, remove the region: 

if [(MGreen - MRed)+(MBIue - MRed)] < -5 remove region. 

c. If the found region is too blue, remove the region: 
if (MBlue > MGreen +15) remove region. 

9.The final green-bluish fluid and foam mask, FGBMaski is calculated by 
performing a flood-fill of "on" valued regions of GBMask firom step 5 with 
seeds in the validated regions firom step 6 and step 7. 
r06231 Next, the method 2258 m figure 1 12 includes processing and segmenting pure white 
regions in steps 2278, 2280, 2282, 2284, 2286, 2288, and 2290. These steps are performed as 
follows: . 

1 .Retrieve glare mask, glareMsk and ROI mask, ROIMsk. 

2. Fill glare regions of WImg using glareMsk to perform run-length boundary 

interpolation as follows: 

a. Raster scan each row of glareMsk to find all beginnings and ends of 
pixel runs. 

b. For each pixel P(x,y) in a given run specified by beginning point P(xb, y) 
and end point P(xe,y) m the intensity image, replace P(x,y) by half the 
linearly interpolated value at P(x,y) from P(xb,y) and P(xe,y). 

c. Raster scan each column of glareMsk to find all beginnings and ends of 
pixel runs. 

d. For each pixel P(x,y) in a given run specified by beginning point P(x, yb) 
and end point P(x,ye) in the intensity image, add to P(x,y) half the linearly 
interpolated value at P(x,y) from P(x,yb) and P(x,ye). 

3. Compute WImg mean, mWlmg, and standard deviation, stdWImg. 

4. Eliminate low intensity areas: 

if WImg < mWlmg - 0.1* stdWImg, WImg = mWlmg - 0.1* stdWImg. 

5 . Rescale the WImg to [0, 1]: 
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WImg =Wlmg -min(Wlmg))/(3.min(Wlmg). 
S.Compute thresholds from image statistics and perform thresholding: 

a. Compute image mean intensity, Imean, for WImg > 0. 

b. Compute image standard deviation of intensity, IstdDev, for WImg > 0. 

c. Compute threshold thW, 

thW = imean +l.io * IstdDev. 

d. Threshold to get the mitial green-bluish fluid and foam mask WMask: 
if ((WImg > thW) AND (pixel is included in ROIMsk)), then 

WMask =^1; 
else 

WMask = 0. 

7.Perform morphological processing to fill small holes and smooth boundaries of 
the found regions in WMask: 

a. Erode the segmentation mask WMask twice^ WMask = erode(WMask, 2). 

b. Dilate the resultant mask three times, WMask = dilate(WMask, 3). 
S-Perform binary region labeling and small region removal: 

a. Perform a connected components labeling, as described, to label all found 
regions. 

b. Compute each region area, area. 

c. Remove small regions that are not likely to the fluid and foam regions: 

If (area < 300) remove the region from the region list. 

9. Whiteness feature validation for each found region based on the original RGB 
values: 

a. For each foimd region, retrieve the mask, iMsk, and compute the mean intensities 
within the region for each of the red, green and blue channels as iMRed, 
iMGreen and iMBIue. 

a. Dilate iMsk five times to obtain iDI Msk = dilate(iMsk, 5). 

b. Compute the perimeter pixels IPeriMsk from iDI Msk: 
fPeriMsk = not (erod (IDIMsk, 1)) AND (IDIMsk)), 1). 
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c. Dilate iPeriMsk three times to get the outer mask: iD2Msk== dilate (iPeriMsk, 
3). 

d. Compute mean intensities on iD2Msk for each of the R, G and B channels as 
perimeter (Outer) means: pMRed, pMGreen and pMBIue. 

e. Compute tiie bmer region gieen-blueness: 
innerGB = (iMGreen - iMRed)+( iMBlue - iMRed). 

f . Compute the Inner region whiteness: 

innerW 3.0 - (abs((iMRed - iMGreen)/( iMRed + iMGreen)) + 

abs((iMGreen - iMBiue)/( iMGreen + iMBlue)) +abs((iMBIue 
- iMRed)/( iMBlue + IMRed))). 

g. Compute the Outer region whiteness: 

outerW » 3.0 - (abs((pMRed - pMGreen)/( pMRed + pMGreen)) + 
abs((pMGreen - pMBIu6)/( pMGreen + pMBlue)) + 
abs((pMBIue - pMRed)/( pMBlue + pMRed))). 

h. Compute tiie Outer region redness: 

outerRed = (pMRed - pMGreen)+( pMRed - pMBlue). 

i. Apply general whiteness validation rule: 

if (((InnerGB < 10) AND (outerRed > 25)) OR (outeiW > (innerW - 
0.1)), then: 

set isFluld to 0, since it is not likely to be a fluid and foam region; 

else, 

SetlsFluidtoL 

j. Very white fluid-foam validation rule: 

If ((innerW > (outerW + 0.16)) set isFluid to 1. 

k. Very high inner green bluish fluid-foam validation rule: 

If (innerGB > 10) set isFluid to 1. 

1 0. The final white fluid-foam mask fWMask is calculated by performing a 
flood-fill of "on" valued regions of Mask from step 8 with seeds in the 
validated regions (IsFluid = 1) from step 9. 
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[0624] Finally, fhe method 2258 in Figure 1 12 includes constructing the final fluid-foam mask. 
The final fluid-foam mask is a logical "OR" of the two segmented and validated masks as 
follows: FluidFoamMask = fBGMask OR fWMask. 

Classifiers 

S [0625] In one embodiment, the tissue characterization system 100 of Figure 1 comprises using 
broadband reflectance data obtained during a spectral scan of regions (interrogation points) of a 
tissue sample to determine probabilities that a given region belongs in one or more tissue- 
class/state-of-health categories. In one embodiment, probabilities of classification are 
deterauned as a combination of probabilities computed by two different statistical classifiers. 

10 The two classifiers are a DASCO classifier (discriminant analysis with shrunken covariances), 
and a DAFE classifier (discriminant analysis feature extraction). The DASCO classifier (step 
1484, Figure 74) uses a principal component analysis technique, and the DAFE classifier (step 
1482, Figure 74) uses a feature coordinate extraction technique to determine probabilities of 
classification. 

IS [0626] The embodiment shown in Figure 74 applies a necrosis mask 1424 and a hard 

*'indetemunate" mask 1426 to a set of arbitrated broadband spectral data to eliminate the need to 
fiirther process certain necrotic and indeterminate interrogation points in the classification steps 
1482, 1484, 1486. After determining statistical classification probabilities in step 1486, the 
embodiment of Figure 74 applies a soft "indeterminate" mask 1428 as well as the NED (no 

20 evidence of disease) classification result 1430 in order to obtain a final characterization 1432 of 
each interrogation point on the tissue sample as Necrotic, CIN 2/3, NED, or Indeterminate. 
[0627] The statistical classifiers in steps 1482 and 1484 of Figure 74 each detemiine respective 
probabilities that a given region belongs to one of the following five tissue-class/state-of-healfh 
categories: (1) Normal squamous (Ns), (2) CIN 1 (Ci), (3) CIN 2/3 (C23), (4) Metaplasia (M), 

25 and (5) Normal columnar (Coi) tissue. Other embodiments use one or more of the following 
tissue classes instead of or in addition to the categories above: CIN 2, CIN 3, NED (no evidence 
of disease), and cancer. The category with the highest computed probability is the category that 
best characterizes a given region according to the classifier used. In one alternative embodiment, 
other categories and/or another number of categories are used. The results of the two statistical 

30 classifiers are combined with the NED mask classification, along with the hard and soft 
'Indeterminate" masks, to obtain a final characterization for each interrogation pouit 1432. 
[0628] In one embodiment, statistical classification includes comparing test spectral data to 
sets of reference-spectral data (training data) representative of each of a number of clashes. A 
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10 



15 



20 



collection of reference spectra fiom the same tissue class is a class data matrix. For example, a 
class data matrix 3} comprising reference spectra (training data) from samples having known 
class j is expressed as in Equation 96 as follows: 

■5,(A,) SM ... SMp) 













_S„/A) 



(96) 



5„,(A) 5„/A,) .-. S„^(A,) 

where class j contains nj referance spectra, S(A), and each reference spectra, S(A) = [S(A,/),S(X,i), 
. . . , S(Aj,)], is a p-dimensional vector where p is the number of wavelengths in a measured 
spectrum. The class data matrix 7} has associated with it a class mean vector fjj (a 1-by-p vector) 
and a class covariance matrix Cj (a p-by-p matrix) as shown in Equations 97 - 99 as follows: 



^tsM -i-i;s,(^) ... -i;s*(A,) 



T 



(97) 



""A A«l 

^-ri(Tj-Mjf(Tj^Mj) 



(98) 



(99) 



Statistical tissue classification uses reference data to determine for a given test spectrum to 
which class(es) and Avith what probabilit(ies) that test spectrum can be assigned. 
[0629] The broadband data used in the statistical classifiers in steps 1482 and 1484 are 
wavelength truncated. For Ihe DASCO classifier (step 1484), only trainic^ data and testing data 
that corresponds to wavelengths between about 400 nm and about 600 nm are used. For the 
DAFE classifier (step 1482), only training data and testing data that correspond to wavelengths 
between about 370 nm and about 650 nm are used. One alternative embodiment uses different 
wavelength ranges. The training data include reference broadband reflectance data from 
interrogation points having a known classification in one of the five states of health, and the 
testing data include broadband reflectance data from a region having an unknown classification. 
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[0630] The discriminant analysis feature extraction (DAFE) method of step 1482 in Figure 74 
transforms a measurement of high dimension into a feature space of lower dimension. Here, the 
feature space is the orthogonal projection in the direction of maximal data discrimination. The 
DAFE method includes constructing feature coordinates by computing the feature space 
S projection matrix. The projection matrix requires the inversion of the pooled within-groups 
covariance matrix, Cpooh Where Tj, 7]?, . . are training matrices for classes 1 through g (here, 
for example, g - 5), the nimiber of reference spectra in a given class, nj, may be less than the 
number of wavelengths in a measured spectrum, p; and Cpooi is therefore singular and cannot be 
inverted, 

ID [0631] Thus, in one embodiment of the DAFE method of step 1482, the spectral measurements 
are subsampled so that a covariance matrix can be computed. In one embodiment, a 
subsampling mte, Uz, is determined according to Equation 100: 



"nearest integer" function. Typically, Uz = 2 or 3, but values up to about 10 do not generally 
remove too much information from a measured reflectance spectrum, and may also be 
considered. After subsampling, the non-singular pooled covariance matrix, Cpooii is computed 
according to Equation 101 as follows: 



where Uk is the number of reference spectra in class k; and Q is the covariance matrix for class k. 
Then, the between-groups covariance, C^/iw, is computed according to Equation 102: 




(100) 



20 



(101) 





g 




(102) 



n 
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[0632] Next, the maxtrix P - • is formed and singular value decomposition is 
applied to obtain the foUovving: 

P=UDV^ (103) 
Let Ug-j equal the first g -1 columns of the orthogonal matrix of singular values U as follows: 



1/ = 



«2I «22 



U 



P>P. 



«U «12 - 



«2I «22 



U-1 



1/ 



(104) 



Then, the feature projection, mapping measured space into feature space, is obtained via right- 
multiplication by Ug-j. 

[0633] The DAFE classification algorithm (step 1482 of Figure 74) proceeds as follows. Let 

AAA A 

, , • • • , be the wavelength reduced, subsampled training (class data) matrices and S(^) be 

A A 

10 the corresponding waveleng& reduced, subsampled test spectrumu The matrices Tj and S(A.)are 
projected into feature space as follows: 



(105) 



Tj^Tj.U,.,^Vj 
S(A)h^S(A)«i7,.iSX 

Next, His ffo\xp mean vectors, groiq) covariance matrices, and pooled wilfain-groups covariance 
matrix are computed using the projection matrix, Vj, in Equation 105, and using Equations 97, 
IS 98, and 101 as shown in Equations 106-108: 

fil^meaniyj) (106) 
Cj = cov(yj) (107) 



(108) 



Then, the Friedman matrix is calculated using the Friedman parameters ;^and A according to 
20 Equation 109 as follows: 



In one embodiment, y= 0 and A = 0.5. Next, the Mahalanobis distance, dj{x), is determined from 
the test spectrum to each data class according to Equation 1 10: 

d^jix) = {x-Mj)*Fyj'M*(^-Mjy (110) 
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The Mahalanobis distance is a (1-by-l) number. Next, the Bayes' score is computed according 
to Equation 111: 

6r/x) = rf;(x)-2to(r^) + hi|det(Fr/y,;i)^ (111) 

The index j at which the minimum Bayes' score is attained indicates the classification having the 
highest probability for the test point in question. ITie DAFE probability of classification for 
class j can be computed for any of the g classifications according to Equation 1 12: 



exp 

Prob(x6ClassJ) = -T— Ml ^= / ' ' " -r—, "T ("2) 



[0634] DAFE classification probabilities are computed thusly for each of the interrogation 
points havmg a test reflectance spectrum, S(^), that is not eliminated in the Necrosis masking 
10 step (1424) or the hard "indetermmate'* maskmg step (1426) in the embodiment shown in Figure 
74. 

[0635] Step 1484 in Figure 74 is the DASCO (discriminant analysis with shrunken 
covariances) method. Like the DAFE method of step 1482, the DASCO metitiod reduces the 
dimensionality of the measured space by transforming it into a lower dimensional feature space. 

15 DASCO differs from DAFE in that the feature space for the DASCO method is along orthogonal 
directions of maximal variance, not (necessarily) maximal discrimination. Also, DASCO uses 
two Mahalanobis distances, not just one. The first distance is the distance to feature centers in 
primary space and the second distance is the distance to feature centers in secondary space. 
[0636] In one embodiment, tiie DASCO method (step 1484) proceeds as follows. First, a 

20 collection {Tu ^2, • • T^} of nj-by-p training matrices is obtained firom reference (training) 
broadband arbitrated reflectance measurements. The amount of reflectance spectral data 
obtained firom a test region (interrogation point), as well as the amount of training data, are 
reduced by truncating the data sets to include only wavelengths between 400 nm and 600 nm. 
[0637] Next, the training data and test data are scaled usmg mean scaling (mean centering) as 

25 follows: 



h^(r^~M^)s7y,whereM^ = 



Ay 



(113) 



S(A) i-> S(;i)-/^ sS/ (114) 
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wliere j = 1, 2, g and g is the total number of tissue-class/state-of-health classes. The number 
of principal components in primary space is %, and the number of principal components in 
secondary space is Uj. The total number of components is nt. In one embodiment, Up = 3, nj = 1, 
and Ht = 4. 

5 [0638] Nejd, the first «/ principal component loadings and scores are computed. This involves 
computing the singular value decomposition of the mean scaled training data matrix Yj firom 
Equation 113, as follows: 

Yj'^UjDjVj (115) 
A similar computation was made in Equation 104. Let Vj,„^ be the matrix comprised of the first «, 

10 columns of Vj. The loadings and scores for Yj are therefore indicated, respectively, in Equations 
116 and 1 17, as follows: 

Uj=Vj^ (116> 

scj'-Yj»Vj,„mYj»Ldj (117) 

15 where Ldj is a p-hy-rii matrix, and scj is an n,-by-«< matrix. 

[0639] The next step in the DASCO method is to compute the class mean scores and 
covariances. First, the class mean vector in primary space, Vjj,, and the class mean vector in 
secondary space, Vj^ are computed as follows: 

Vj = mem(scj) (the mean is computed analogously to //y in Equation 97) (118) 

20 Vy s[vj,„Vj.2,-,Vj„^,V^^^,.V^^^,,-,Vj^^,„J ^ K,,V,J = V^., © V^^ (119) 

where Vj,^ =[vj,i,Vj,j,-,Vj^J and =t^j,^^,,Vj^^.-,v^^^^,J (120) 

Next, Cj - cov(scj) is defined as the class covaiiance matrix analogous to that in Equation 100. 
In a manner similar to tiie computation of Ae primary and secondary space class mean vectors 
above, Cj is decomposed into the primary (Cj,p) and secondary (Q,) space covariance matiices 
25 according to Equations 121 - 124 as follows: 

Cj=cj,ecj, (121) 



CxxU) c,^,C/) <^M,«C/) - Cu>,C/)" 

C21U) CnU) - Cj^,C/) C2^,*lU) - C2^,*n.U) 

. • , . • • 

: : : 

^.lOO s.2C/> - S.»,+iCO s^,+20*) ^^,+».0) 



(122) 
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20 



CjiO) c^U) - CO 



(123) 



U) 

• • • 

• • • • 

• • • 

(7)1 



(124) 



[0640] Next, tbe scaled test spectrum &om Equation 1 14 is projected into each principal 
component space according to Equation 125: 

x(0 = M-»Sy (125) 
Then, x{f) is decomposed into primary and secondary space vectors as follows: 

x(0 = M'), X2(/") Xnjif)] ^^j,® X,.. (126) 



where Xyj, = [x;(/)» X2(0. • • x„^(/)] is the projection of x(/) into primary space and X/,, = [x„^ 



0% 



X« +20')» •» x« +fl (/)] is the projection of x(0 into secondary space. 
P P ^ 

10 [0641] The MahdanoUs distances in primary and secondary space are computed according to 
Equations 127 and 128 as follows: 

dlMuM^P •(x,., -v,.,)^ (127) 
^L(xa)) = (x,, - v,J*F,:J .(x,, - v,^)^ (128) 

where = — *^„,xn. • Then, the total distance is computed according to Equation 129 as 

IS follows: 

rf(xC/))= 4dlA^(J))^-dl{x(j)) (129) 

[0642] The DASCO probability of class assignment to classj is obtained by computing the 
Bayes' score according to Equations 130 and 131 as follows: 

Mxa))=</HxO'))-21n(r,)+hifdet(Cy.,)|)+n, .hi|det(Jf>vj|) (130) 

Prob(x(/-) e Classy) = , \ 

5^exp^-^6r,(x(fc))J 



(131) 
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Equation 13 1 is evaluated for all ciassesj = 1, 2, . , . DASCO classification probabilities are 
computed thusly for each of the interrogation points havir^ a test reflectance spectrum, S(;i), that 
is not eliminated in the Necrosis masking step (1424) or tiie hard "indeterminate** masking step 
(1426) in the embodiment shown in Figure 74. 

[0643J Probabilities determined using the DAFE classifier m step 1482 of Figure 74 and 
probabilities deteraiined using the DASCO classifier m step 1484 are combined and normalized 
in step 1486 to obtain for each interrogation point a set of statistical probabilities that the point 
belongs, respectively, to one of a number of tissue-class/state-of-healtii categories. In one 
embodiment, there are five classes, as described above, including tiie following: (1) Normal 
squamous (Ns), (2) CIN 1 (Ci), (3) CDSf 2/3 (C23), (4) Metaplasia (M), and (5) Columnar (Coi) 
tissue. 

[0644] The probability matrices Pdafe and Pdasco contam probability vectors coirespondmg to 
the interrogation points in the scan pattern and are expressed as shown in Equations 132 and 133 
as follows: 



Pdafe, I (2) Pdafb,i (2) 



Pdafb,sQ^) 

PoAFB^i^) 



PdafbMp) PdafeA^V) Poafb,Mp) 



(132) 



PjllSCO,\0) PoASCOa^^) '•' pDAXO.gO) 

Paisco.i(2) Paisco^(2) ••• PiuscoA^) 



(133) 



PDAsco,\inv) Pdasooa^^P) "' PMscojs(pip)_ 
where g is the total number of classes (for example, g = 5); nip is the total number of 
interrogation points for which DAFE and DASCO probabilities are calculated (for example, nip 
= up to 499); Pdaew 0) represents the DAFE probability that the interrogation point j belongs to 
class i; and Pdascoj (j) represents the DASCO probability that the interrogation pomt j belongs 
to class i. 

[0645] Step 1486 of Figure 74 represents the combination and normalization of classification 
probabilities determined by the DAFE and DASCO classifiers in steps 1482 and 1484, 
respectively. Hie combined/normalized probability matrix, PcouBi is obtained by multiplying 
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the probability matrices Pdafe and Pda^o (Equations 134 and 135) element-wise and dividing 

the row-wise product by the sum of each row's elements. 

Combining spectral and image data 

[0646] The block diagram of Figure 74 includes steps representing the combination of spectral 
5 masks and image masks (1468, 1470, 1472, 1474), as well as the application of the combined 

masks (1466, 1476, 1424, 1478, 1480, 1424, 1426, 1428, 1430) m a tissue characterization 

system, according to one embodiment. These steps are discussed in more detail below. 

[0647] As discussed above, the Necrosisspec mask identifies interrogation points whose spectral 

data are indicative of necrotic tissue. Since necrosis is one of the categories in which 
10 interrogation points are classified in step 1432 of Figure 74, the Necrosisspcc mask is used not 

only to eliminate interrogation points from further processing, but also to positively identify 

necrotic regions. Therefore, it is necessary to filter out points affected by certain artifacts that 

may erroneously cause a positive identification of necrosis. 

[0648] Step 1466 of Figure 74 indicates that two image masks are applied to the necrosis 

15 spectral mask - the smoke tube mask, [STjvid, 1450 and the speculum mask, [SPjvid 1452. 

Regions in which a speculum or smoke tube has been identified cannot be positively identified as 
necrotic. Thus, interrogation points having any portion covered by pixels indicated by the smoke 
tube mask, [STjvid, 1450 and/or the speculum mask, [SP]i^d,1452 are identified as 
"Indeterminate*' and are eliminated fi-om the necrosis mask. 

20 [0649] Following this treatment, the necrosis mask is then applied in the broadband reflectance 
spectra classification sequence in step 1424 of Figure 74, Each interrogation point at which the 
necrosis mask applies is classified as '"Necrotic'*. The broadband spectral data at these 
interrogation points are then eliminated firom further processing, or, alternately, the results of the 
statistical classifiers at these points are ignored in favor of classification of the points as 

25 "Necrotic". Shnilarly, the necrosis mask is applied in the NED (no evidence of disease) spectral 
classification sequence in step 1476 of Figure 74. Each interrogation point at \\ddch the necrosis 
mask applies is classified as 'TSTecrotic". The NEDspec mask need not be computed for these 
interrogation pomts, or, alternately, the results of the NEDspec mask at these pomts may be 
ignored in favor of classification of the points as '^Necrotic". 

30 [0650] Three image masks are combined to form a fluorescence hard mask, "F Hard," which is 
applied in the NED (no evidence of disease) spectral classification sequence in step 1478 of 
Figure 74. As discussed hereinabove, hard masking results in a characterization of 
"Indeterminate" at affected interrogation pomts, and no fiirther classification computations are 
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necessary for such points. The combined fluorescence hard mask, Hard," 1468 is a 
combination of the three image masks shown in Figure 74 (1448, 1450, 1452), according to 
Equation 134 as follows: 

F Hard = [ROI]vid OR [ST]vid OR.[SP]vid (134) 

5 The combined "F Hard" mask is applied in the NED spectral classification sequence in step 1478 
of Figure 74. Each interrogation point at which the Hard" mask applies is classified as 
''Indetemiinate" The NEDspec mask is not computed for these inteirogation points. The'T 
Hard" mask applies for each interrogation point having any portion covered by pixels indicated 
by the "F Hard" combined image mask. 

10 [0651] Two spectral masks and five image masks are combined to form a broadband 

reflectance "hard" mask, which is applied in the broadband reflectance statistical classification 
sequence in step 1426 of Figure 74. The combined hard mask, "BB Hard", 1474 uses the image 
masks [STJvid, [SPJvid, [R0I]vi6 and [VW]vid (1450, 1452, 1448, 1454) as hard masks, and also 
treats them as "anchors" to qualify the sections of the two spectral masks - [CE]spec and [MU]spec 

15 (1444, 1446) - that are used as hard masks. The outer rim of interrogation points in the spectral 
pattem is also used as an anchor to the spectral masks. Finally, the intersection of the fluid-and- 
foam image mask [FL]vid (1456) and the mucus spectral mask [MU]spec (1446) is determined and 
used as a hard mask in "BB Hard" (1474). Each interrogation point at which the "BB Hard" 
mask applies is classified as "Indeterminate". The broadband spectral data at these interrogation 

20 points are then eliminated firom further processing, or, alternately, the results of the statistical 
classifiers at these points are ignored in favor of classification of the points as ^Indeterminate". 
[0652] In one embodiment, the combined hard mask, "BB Hard," 1474 of Figure 74 is 
determined according to the following stq)s. 

[0653] First, form a combined image processing hard mask IPHardlPMsk using all the 
25 mterrogation points (IP's) that have any portion covered by one or more of the foUowng image 
masks: [STJvid, [SP]vid, [VW]vid and [ROI]vid. The combined mask is expressed as: 
IPHardlPMsk = [ST]vid OR [SP]vid OR [VW]vid OR [ROI]vid. Extend IPHardlPMsk to 
include the level one and level two neighbors of the mterrogation points mdicated above. For 
example, each IP that is not on an edge has 6 level one neighbors and 12 level two neighbors, as 
30 shown in the scan pattem 202 in Figure 5. LetextlWIHardlPMskbe thenewmask. Add all 
outer rim interrogation points to extlMHardlPMsk to form anchorMsk. The rim is defined by 
tiie following interrogation points for the 499-point scan pattem 202 shown in Figure 5: 1-9, 17- 
20, 31-33, 47-48, 65-66, 84-85, 104-105, 125-126, 147-148, 170, 193, 215-216, 239, 263,286- 
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287, 309, 332, 354-355, 376-377, 397-398, 417-418, 436-437, 454-455, 469-471, 482-485, 493- 
499. Form a combined spectral mask SpeclPNIsk using all the interrogation points tliat are 
marked as either [CE]spec or [MU]spec (or both). Intersect the image processing anchor mask and 
the combined spectral mask to obtain SPHardMsk: SPHatdMsk= anchorMsk AND 
SpecIPMsk. Intersect the im&g& processing mask, |TFL]vid, and spectral mucus mask, PvllJ|spec to 
obtain the fluid hard mask FluidHardlPMsk. FluidHardlPMsk = {FLlfa AND ([MU],pecOR 
[CE] spec). Finally form the final hard mask: BBHard = IPHardlPMsk OR SPHardMsk OR 
FluidHardlPMsk. 

[0654] Two image masks - Bloodvid and Osvid (1458, 1460) - are combined to form a 
fluorescence "soft" mask, "F soft," 1470 \\diich is applied in the NED spectral classification 
sequence in step 1480 of Figure 74. As (Uscussed hereinabove, soft masking involves applying a 
v^eighting function to data fi-om points identified by Has mask in order to weight the data 
according to the likelihood they are affected by an artifect. The mask "F soft" determines two 
weighting fimctions - penwoodCIP) and peoJJP) - for interrogation points (IP's) that are at least 
partially covered by the image masks Bloodvid and Osvjd (1458, 1460). As discussed 
heremabove, a percentage coveri^e, o, is determined for each interrogation point according to 
the percentage of pixels corresponding to the mterrogation point tiiat coincide wilh the image 
mask. For the image masks Bloodvid and Osyid, (1458, 1460), corresponding values abiood(IP) and 
OosCfP) are determined for each affected interrogation point, and Equations 135 and 136 are used 
to calculate the corresponding weigihting at these interrogation points: 

penbiood(IP) = 1 - awood(IP) (135) 
penos(IP) = l-aos(IP) (136) 
The plication of penMood(IP) and pen«,(IP) in the NED spectral classification sequence of step 
1480 is discussed in more detail below. 

[0655] Two unage masks - Glarevw and Mucusvid (1462, 1464) - are combined to form a 
broadband reflectance "soft" mask, "BB soft", 1472 >^ch is applied m the broadband 
reflectance statistical classification sequence in step 1428 of Figure 74. As discussed 
hereinabove, soft masking involves applying a weighting fimction to data firom points identified 
by Ihe mask in order to weight the data according to the likelihood it is affected by an artifect. 
The mask "BB soflf' detramines two weighting fimctions - pengiare(IP) and penmucus(IP) - for 
interrogation points (IP's) that are at least partially covered by the unage masks GlarevSd and 
Mucusyid (1462, 1464). As discussed hCTemabove, a percentage coverage, o, is determined for 
each interrogation point according to the percentage of pbcels correspondmg to the interrogation 
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point that coincide with the image mask. For the image masks Glareyid and Mucus^d, (1462, 
1464) corresponding values OgiareCIP) and OmucusCIP) are determined for each affected 
interrogation point, and Equations 137 and 138 are used to calculate the corresponding penalties 
at these interrogation points: 



Hie application of peng|aic(IP) and penmucus(IP) in the broadband reflectance statistical 

classification sequence at step 1428 is discussed in more detail below. 

[0656] The tissue-^lass/state-of-health classification of interrogation points includes the 

10 application of masks as determined above. These steps are shown in Figure 74. The tissue- 
class/state-of-health classification method includes an NED (no evidence of disease) spectral 
classification sequence, as well as a broadband reflectance statistical classification sequence, that 
apply the combined hard masks and soft masks described above. As discussed hereinabove, the 
separate identification of necrotic regions and NED regions based on at least partially heuristic 

15 techniques allovsrs for the development of a statistical classifier that concentrates on identifying 
tissue less conducive to heuristic classification, for example, CIN 2/3 tissue. Furthermore, by 
eliminating data affected by artifacts, the statistical classifiers are further improved, leading to 
improved sensitivity and specificity of the final classification of a tissue sample. 
[0657] The Necrosis mask (1424, 1476), "BB Hard" mask (1426), and "F Hard" mask (1478) 

20 are applied as shown in Figure 74. Interrogation points coinciding with these masks are 
identified as either ^^Necrotic" or "Indeterminate'', as discussed hereinabove. In one 
embodiment, these regions are removed fi^om further consideration. The NED classification 
sequence then applies the Sofif' mask in step 1480. This is performed as explained below. 
[0658] The NEDspec mask identifies interrogation points that indicate normal squamous tissue, 

25 which is class (1) of the five classes used by the DAFE and DASCO classifiers discussed 
previously. The NEDspec mask assigns at each mdicated (masked) interrogation point a 
probability vector = [1, 0, . . ., 0], where the normal squamous classification probability, Ns 
(class 1), is set equal to 1 and all other class probabilities are set equal to 0. The "F Soft** mask 
is applied in step 1480 by multiplying the Ns probability of indicated (masked) NED 

30 interrogation points by the product of the blood and os weighting functions, peUbioodOP) ' 

penos(IP). Hence, the normal squamous classification probability, Ns, at these points will be less 
tiian 1.0. If the product, penb!ood(IP) ' penos(IP), is equal to 0, then the interrogation point IP is 
classified as "Indetemtinate". The NEDspec mask probability vector - 0 for all other 



5 



pengiare(IP)=l-{agiare(IP)} 
penmucus(IP) = 1 - ttmucusCIP) 



1/5 



(137) 
(138) 
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interrogation points. It is noted that if an interrogation point is not identified by the NEDspec 
mask, its Ns probability calculated by the broadband reflectance statistical classification 
sequence is unaffected. The application of the overall NEDj^ mask is explained below in the 
discussion of step 1430 in Figure 74. 

S [0659] The broadband reflectance statistical classification sequence applies the Necrosis mask 
(1424) and tiie "BB Hard" mask (1426) before determining statistical classification probabilities 
in steps 1482, 1484, and 1486. As discussed above, the output of the broadband statistical 
classification is the probability matrix, Pcom, made up of probability vectors for tiie 
interrogation points, each vector indicating respective probabilities that a given interrogation 

10 point belongs to one of the five tissue-class/state-of-health categories - (1) Normal squamous 
(Ns), (2) CIN 1 (Ci), (3) CIN 2/3 (C23), (4) Metaplasia (M), and (5) Colunmar (Coi) tissue. The 
broadband reflectance statistical classification sequence then applies the ^'BB Soft" mask in step 
1428 by multiplying all five probabilities for each affected (masked) interrogation point by the 
quantity pengian!(IP) * p^ucus(IP)* 

15 [0660] Step 1432 of Figure 74 classifies each interrogation point as Necrotic, CIN 2/3, NED, 
or Indeterminate. In one embodiment, the probabilities in Pcom that correspond to CIN 2/3 
classification, pcom,c23(JP) [class 3], are considered indicative of "CIN 2/3" classification in 
step 1432, and all other classification categories in Pcom ~ classes 1 , 2, 4, and 5 (Ns, Ci, M, and 
Col) - are considered indicative of ^'NED" tissue. In an alternative embodiment, further 

20 classification distinctions are made in step 1432. 

[0661] In step 1430 of Figure 74, the results of the NEDspec mask are applied to the broadband 
reflectance-based classifications, Pcom* The '"Necrotic" interrogation points and the hard- 
masked "Indeterminate" points have been identified and removed before step 1430. In step 
1430, the remaining interrogation points are either classified as ''indetemiinate" or are assigned a 

25 value of CIN 2/3 classification probability, pc25(IP)- .Here,/?c2i(IP) is the CIN 2/3 classification 
probability for interrogation pomt IP that is set as a result of step 1 430. Interrogation points that 
are not identified by the NEDspec niask have been assigned NEDspec mask probability vector ps = 
0, and pc23(JP) = Pcom.c23(JP) for these points. Interrogation points that are identified by the 
NED mask have = [1 , 0, . . 0], or ps = [{penbiood(IP) ' penos(IP)}, 0, . . 0], (where (IP) = 

30 1 or penbiood(IP) ' penos(IP)) depending on whether the point has been penalized or not by the "F 
Soft'* mask in step 1480, The following describes how values of paaQP) are determined for 
interrogation pionts that are identified by the NEDspec mask: 
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Due to spectral arbitration in step 128 of Figure 74, the broadband signal may 
have been suppressed for some interrogation points, and only fluorescence spectra 
are available. For these interrogation points, the following rules are applied in 
step 1430 of Figure 74: 
5 1. Wps,us(JP) > 0, THEN pc23(JP) = 0. 

2. ELSE the interrogation point IP is classified as 
"Indeterminate*', 

For points having a valid arbitrated broadband signal and fluorescence signal, the 
following fules are applied in step 1430 of Figure 74: 
10 1. JFps^NsQP) = 1, THESpc23QP) = 0. 

. 2, Wps^Nsm^0,TmNpc23m^PCOMB.C23(JPy 

3, IF psMs(JP) < U THEN: 

JFps^NsQP) <PcomN^)> THENj?«i(IP) = Pcom^asQPl 
ELSE,i7c23(IP) = 0. 

15 [06621 Step 1432 of Figure 74 classifies each interrogation point as Necrotic, CIN 2/3, NED, 
or Indeterminate. Necrotic and hard-masked Indeterminate interrogation points are identified 
prior to step 1430, as described above. In step 1430, the remaining interrogation points are either 
classified as Indeterminate or are assigned a value of pcisQPy For these points, if pcisQP) = 0, 
the pomt is classified as NED. if pc23(JP) > 0, the point is considered to have a non-zero 

20 probability of high grade disease (CIN 2/3). In one embodiment, disease display (step 138 of 
Figure 74) uses these non-zero pc23(J^) values to distinguish regions having low probability of 
CIN 2/3 and regions having high probability of CIN 2/3. 

[0663] Step 1434 of Figure 74 represents post-classification processing. In one embodiment, 
this includes a final clean-up step to remove isolated CIN 2/3-classified interrogation points on 
25 the outer rim of the spectral scan pattern (for example, the outer rim consists of the numbered 
interrogation points listed hereinabove. A CIN 2/3-classified interrogation point is considered 
isolated if it has no direct, level-1 neighbors that are classified as CIN 2/3. Such isolated points 
are re-classified as "Indeterminate" in step 1434 of Figure 74. 

Image enhancement 

30 [0664] The brightness of an acquired image of a tissue sample may change firom patient to 
patient due to obstructions, tissue type, and other factors. As a result, some images may be too 
dark for adequate visual assessment. Step 126 of the tissue characterization system 1 00 of 
Figure 1 performs an image visual enhancement method to improve the image visual quality, 
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using an image intensity transformation method. The improved image may then be used, for 
example, in the disease display of step 138 of Figure 1. 

[0665] In one embodiment, the visual enhancement method of step 126 in Figure 1 involves 
analyzing the histogram of the luminance values of an input image, detemiimng luminance 

5 statistics using only portions of the image corresponding to tissue, and performing a piecevdse 
linear transformation to produce a visually enhanced image. Step 126 involves using die image 
masks, as shown in step 108 of Figures 1 and 73 and as described previously, in order to 
determine which portions of the image are used to compute the image statistics. Step 126 
includes performing brightness and contrast enhancement, as well as applying image feature 

10 enhancement to improve local image features such as edges, borders, and textures of different 
tissue types. Finally, a color balancing correction is applied to reduce the redness in certain 
images. 

[0666] The visual enhancement method of step 126 includes determining which portions of the 
input tissue image correspond to tissue in the region of interest, as opposed to artifacts such as 

15 glare, mucus, a speculum, the os, blood, smoke tube, and/or areas outside the region of interest. 
Only the regions conesponding to tissue of interest are used in determining luminance statistics 
used in performing the visual enhancement In one embodiment, the image masks of Figure 73 
and 74 are used to determine the portion of the image corresponding to tissue of interest In one 
embodiment, this image portion is [tROrjvid, a subset of the [ROrjvid mask, computed in Equation 

20 139 as follows: 

[tROrivid = [ROIJvid - {[Glare]vid + [SPlyid + [os]vid + Bloodvw + Mucusvid + [STlvid} (139) 
where the image masks above are as shown in Figure 74 and as described above. 
[0667] Figure 1 13A-C show graphs representing a step in a method of image visual 
enhancement in which a piecewise linear transformation of an input image produces an output 

25 image with enhanced image brightness and contrast A histogram 2328 is computed for the 

luminance values \x (2326) of pixels within [tROI]vid of an input image, and the histogram is used 
to determine parameters of a piecewise linear transformation shown in the plot 2324 of Figure 
113B. The transformation produces luminance values v (2330) of a corresponding brightness- 
and contrast-enhanced output image. The transformed image generally has a vdder range of 

30 luminance values, stretching from the minimum intensity^ (0) to the maximum intensity (25 S), 
than the input image. The luminance values from the input image are transformed so that input 
luminance values within a given range of the mean luminance are stretched over a wider range of 
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Ihe luminance spectrum than input luminance at the extremes. In one embodiment, tiie 
piecewise linear transformation is as shown in Equation 140: 

v = iP(ji-Ma)+Vo> Ma ^ ft < Ml. (140) 
yiM'Mb) + ^b* Mb^M<L^ 

where Lmax is the maximum luminance value of a pixel within [tROI]vid of the input image; the 
5 parameters Ha» Wj, Va, and Vb are piecewise linear breakpoints; and a,p, and y are slopes of the 
trat^foimation. 

[0668] In one embodiment, the image brightness and contrast enhancement is performed 
according to the following steps. Fkst, calculate the luminance L from the RGB signal of the 
mput image usmg the formula: L = 0.299 * R + 0.587 * G + 0.1 14 * B. Extract the luminance 
10 image LROI within tROI ([tROI]vid): LROI = L AND tROL Compute LROI mean, 

IMean.Compute the piecewise linear breakpoints ma, mb, na, nb (^af V^, Va, and Vb) from the 
LROI histogrami nHist[], as follows: 

1 . If ((IMean > 38) AND (IMean < 132)): 

a. Compute and normalize nHist[ ] to the range [0, 1]. 
15 b. Compute ma and mb, the 5% and 98% histogram tails: 

ma = i, if sum(nHjst [i]) > 0.05, i = 0 1» 255. 
mb = i, if sum(nHist [i]) > 0.98, i = 0 to 255. 

c. Define the expected low and high intensity parameter na and nb: 

d. na = 0andtib»180. 

20 2. If (IMean > 38 AND (IMean < 132) AND ((ma ^ na AND ma < 100 AND nb 

> 20)), compute the slope or the degree of enhancement, bcDOE: 
bcDOE = (nb - na) / (mb - ma). 
3. If ((IMean > 38) AND (IMean < 132)), apply brightness and contrast 
enhancement transfoimation to input color image inRGB to obtain bcRGB 
25 (brightness and contrast enhanced color image). 

[0669] In addition to producing an output image with enhanced image brightness and contrast, 
the visual enhancement method of step 126 (Figure 1) also includes peifomung an image feature 
(local contrast) enhancement of the output image to emphasize higih jfrequency components such 
as edges and fine features for the purposes of visual inspection. In one embodiment, image 
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feature enhaQcement is performed using a spatial filtering technique according to Equations 141 
and 142 as follows: 



where G(m, n) is the gradient image; p is the degree of the enhancement Iin(m, n) and loutOn, n) 
are the original and the resultant image of the feature enhancement operation; 
and S(m, n) is the smoothed Gowpass filtered) version of Iin(m, n). 
[0670] In one embodiment, the image feature enhancement operation of the visual 
enhancement method of step 126 is performed according to Hie following steps: 
IflMean>38: 

1. Smooth bcRGB (brightness and contrast enhanced color image) with a 7x7 
boxcar filter to obtain smRGB. 

2.Subtract smRGB from bcRGB to obtain the gradient image, grRGB. 

3. Dilate glareMsk twice to obtain dGlareMsk ^ dil (glareMsk, 2). 

4. Remove dilated glare regions form gradient image to avoid emphasizing glare regions: 

a. Convert gray image dGlareMsk to RGB image, dGlareMskC. 

b. Remove glare image firom gradient image to obtain grRGBgl: 
grRGBgl = grRGB - dGlareMskC. 

5. Define the degree of feature enhancement, feDOE, fix>m experiments, feDOE = 0.8. 
6.Scal6 grRGBgl by feDOE to obtain feRGB. 

7. Add feRGB to bcRGB to produce image feature enhanced image fRGB. 
[0671] In addition to producing an output image with enhanced image brightness, contrast, and 
image features, the visual enhancement method of step 126 (Figure 1) also includes performing 
color balancing to reduce redness in certain overly-red tissue images, based on a mean-red-to- 
mean-blue ratio. 

[0672] In one embodiment, the color balancmg operation of the visual enhancement method of 
step 126 is performed according to the following steps: 
IflMean>38: 

l.Split RGB (i.e. of the image feature enhanced image fRGB) into R, G, B. 

2. Extract the R image (within the tROIMsk) and compute mean tissue redness, tRed. 



G{m,n) = - S{m,n) 



(141) 



(142) 
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3 . Extract the B image (wifhin the tROIMsk) and compute mean tissue blueness tBIue. 

4. Compute the red-blue ratio as RBRat = tRcd / tBlue. 
5* Perform color balancing: 

If RBRat < 1.20, no red redection. 

5 Elseif RBRat >-1.20 AND RBRat < 132, R = 0.95* R. 

Else if RBRat >=:132 AND RBRat < 1.55, R = 0.90* R. 

Else if RBRat >=1.55, R = 0.85*v. 

6. Combine the R, G and B channels to form the final color image for display. 

Diagnostic display 

10 [0673] In one embodiment, the invention assigns tissue-class probability values to discrete 
regions of a patient sample, and creates an overlay for displaying the results. One feature of the 
overlay is that it facilitates display of the tissue class probabilities in a way that reflects the 
diagnostic relevance of the data. For example, an embodiment of the invention comprises 
applying filtering and color-blending techniques in order to facilitate display of diagnostic 

IS results. These techniques enhance certain portions of tiie overlay in order to highlight 
diagnostically-relevant regions of the sample. 

[0674] Diagnostic evaluation is further facilitated by creation of a composite of the overlay 
map with a reference image of the sample. For example, methods of the invention can represent 
a range of tissue-class probabilities as a spectral blend between two colors that contrast with an 

20 avemge tissue color. In one embodiment, a portion of tiie spectrum representing low probability 
of disease is blended with an average tissue color so that tissue regions associated with a low 
probability of disease are featured less prominentiy in the composite. 
[0675] Preferred embodiments of the invention comprise application of diagnostic data that 
properly account for indeterminate regions of a tissue sample. A region may be diagnosed as 

25 indeterminate if it is affected by an obstruction or if it lies outside a zone of diagnostic interest 
A region of a tissue sample may be obstructed, for example, by mucus, fluid, foam, a medical 
instrument, glare, shadow, and/or blood. Regions that lie outside a zone of interest include, for 
example, a tissue wall (e.g., a vaginal wall), an os, an edge surface of a tissue (e.g., a cervical 
edge), and tissue in the vicinity of a smoke tube. Data masking algorithms of the invention 

30 automatically identify data firom regions that are obstructed and regions that lie outside a zone of 
interest based on spectral data obtained ftom those regions. In one embodiment, tiie overlay 
identifies indeterminate regions without obscuring corresponding portions of the reference 
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image, when vie>ved as a composite. Similarly, necrotic regions may be indicated on the 
overlay, according to results of necrotic data masking algorithms of the invention. 
[0676] Systems of the invention allow performing fast and accurate image and spectral scans 
of tissue, such that both image and spectral data are obtained from each of a plurality of regions 

5 of the tissue sample. Each data point is keyed to its respective region, and the data are used to 
determine tissue-class probabilities for regions of interest, as well as to identify indeterminate 
riegions* These systems allow real-time display of diagnostic results during a patient 
examination. For example, data may be obtained from an in vivo tissue sample, and results of a 
tissue classification algorithm may be displayed either during or immediately following the 

10 examination. This provides a medical professional with nearly instantaneous, feedback which 
may be quickly comprehended and used for continued or follow-up examination. In some cases, 
the display is prepared within seconds of obtaining data from the tissue. In other cases, the 
display is ready within a matter of minutes or a matter of one or more hours after obtaining data 
from the tissue. 

IS [0677] Accordingly, the invention comprises providing tissue-class probabilities corresponding 
to regions of a tissue sample, creating an overlay that uses color to key the probability values to 
the corresponding regions, and displaying a composite of a reference image of the tissue sample 
with the overlay. Methods of the invention pref^ntially include color-blending and/or filtering 
techniques designed to convey diagnostically-relevant data in a manner commensurate with the 

20 relevance of the data. In one embodiment, methods of the invention are performed such that 
diagnostic results are displayed in real-time during a patient examination. The step of providing 
tissue-class probabilities may comprise actual determination of diagnostic results according to 
methods of the invention. Alternatively, simply supplying probability values obtained using any 
tissue classification method may encompass the providing step according to the invention. 

25 [0678] In one embodiment, the tissue characterization system 100 of Figure 1 comprises 
producing a disease probability display 138 for a reference (base) image of a test tissue sample 
using the interrogation point classifications in step 1432 of Figure 74 - Necrotic, CIN 2/3, NED, 
and Indeterminate. A method of disease probability display 138 includes producing an output 
overlay image with annotations for indeterminate regions, necrotic regions, and/or regions of 

30 low-to-high probability of high-grade disease, according to the classifications determined in step 
1432 of Figure 74 for a given patient scan. The annotations are shown as an overlay on top of 
the reference tissue image to provide easily-discernible tissue classification results, for example. 
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indicating regions of concern for the purposes of biopsy, treatment, diagnosis, and/or further 
examination. 

[0679] In one embodiment, indeterminate regions are indicated by a gray "see-through" 
Crosshatch pattem that only partially obscures the underlymg reference image. Necrotic regions 
5 are indicated by a green trellis pattem. Regions of tissue associated with high-grade disease (for 
example, CIN 2/3) are indicated by patches of contrasting color which intensify according to the 
likelihood of high-grade disease. 

[0680] In one embodiment, the disease probability display method 138 of Figure 74 as applied 
to a reference image of tissue from a patient scan includes the following steps: determining a 

10 disease display layer from the classification results of step 1432, overlaying the disease display 
layer on the reference image, deteraiming an "mdeterminate" mask from tiie classification 
results, overlaying the indeterminate mask on the disease display image using a gray Crosshatch 
pattern, determining a "necrosis" mask from the classification results, and overlaying the 
necrosis mask on the disease display image using a green trellis pattem. The result of the disease 

15 probability display method 138 of Figure 74 is a state-of-healtii ^Wp" of the tissue sample, with 
annotations indicating indeterminate regions, necrotic regions, and/or regions of low-to-high 
probability of high-grade disease. 

[0681] Figure 1 14B represents an exemplary image of cervical tissue 2358 obtained during a 
patient examination and used as a reference (base) image in constructing an output overlay 

20 image in the disease probability display method 138 in Figure 74. Figure 1 14B shows the output 
overlay image 2360 produced by the disease probability display method 138 in Figure 74 that 
corresponds to the reference hnage 2358 in Figure 1 14A. The output overlay image 2360 in 
Figure 1 14B contains annotations indicating indeteiminate regions (2366), regions associated 
with a low probability of CIN 2/3 (2362), and regions associated with a high probability of CIN 

25 2/3 (2364). 

[0682] The disease probability display method 138 begins with the determination of a disease 
display layer from the CIN 2/3 classification results of step 1432 in Figure 74. In step 1432, 
values of pcisO^) are determined for interrogation points having a non-zero probability of high- 
grade disease (here, CIN 2/3). An area of tissue indicative of high-grade disease is represented 
30 on the disease display layer as an area whose color varies fix)m yellow-to-blue, depending on 
values of pc23(JP) at corresponding interrogation points. The yellow color represents low 
probability of high-grade disease, and the blue color represents high probability of high-grade 
disease. At the low end of tiie probability range, the yellow color is blended into the reference 
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image so that there is no sharp discontinuity between the high-grade disease region and the 
image. In one embodiment, a minimvim cut-off probability, pc23minO^\ is set so that 
interrogation points with values of pc23Q^) lower than the minimum cut-off do not show on the 
disease display layer. Inoneembodiment,/7c?5m2n(IP) = 0.2. 

S [0683] Figures USA and 1 15B represent two stages in the creation of a disease display layer, 
according to one embodiment. Figure 1 ISA shows the disease display layer 2368 v^erein high- 
grade disease probabiliti^ are represented by circles with intensities scaled by values of pc23(fP) 
at corresponding interrogation points. In order to more realistically represent regions of high- 
grade disease on the tissue sample, the circles in Figure 1 ISA are replaced vrith cones, then 

10 filtered to produce the disease display layer 2372 shown in Figure 1 1 SB. 

[0684] Finally, the grayscale intensity values are converted to a color scale so that regions of 
high-grade disease appear on the overlay image as patches of contrasting color that intensify 
according to the likelihood of disease. 

[0685] In one embodiment, the disease probability display method 138 of Figure 1 includes 
15 creating a disease display layer according to the following steps: 



1 .Retrieve the reference image (base image). 



2.1f all IPs are indeterminate, skip to creating the Indeterminate Mask. 



3.Generate CIN 2/3 probability image, Ip, of base image size, for all non- 
indeterminate IPs: 



20 



Generate a regular truncated cone centered at (15,1S) on a square matrix 
of size 29-by-29, set to 0: 

i. The two truncating circles are centered around (1 S,l S) and have a 
radius Ro = 14 and R| = 6. 



ii. For each cone point, cone(i, j), let R be the distance from the 



25 



geometric center (IS, IS). 

1. IfR>Ro,cone(i,j) = 0. 

2. IfR<R|,cone(i,j) = l. 

3. If Ri R <= Ro, cone (i, j) = (Ro - R) / (Ro - Ri). 



30 



c. 



b. 



Initialize Ip to 0. 

For each IP with probability pc23QP) > 0.2: 
L make a copy of the cone; 
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ii. scale it by p; 

iii. add it to tp with the cone* s center aligned with the IP location. 

d. Smooth Ip using a 33 by 33 separable symmetric Hamming window filter 
specified by: 

i. the following coefficients (since the filter is symmetric around the 
origin, only 17 coefficients are specified below; the others are the 
mirror image around 1 .0): 

(0.0800, 0.0888, 0.1150, 0.1575, 0.2147, 0.2844, 0.3640, 0.4503 

0.5400, 0.6297, 0.7160, 0.7956, 0.8653, 0.9225, 0.965, 
0.9912,1-0); 

ii. a gain of (0.85/301. 37) for the 33 point ID filter, 

e. Linearly rescale Ip fi*om the [0.2 1] range to the [0 1] range. 
£ Clip rescaled Ip to range [0 1]. 

4.Compute an RGB colormap image and an alpha blending channel from the 
probability image Ip. The colormap defines a transformation from integer 
intensity values in the range [0,255] to an RGBa image. 

a. The R colormap is a piecewise Imear map specified by the following 
breakpoints [0,255], [97,220], [179,138] and [255,0]. 

b. The G colormap is a piecewise linear map specified by the following 
breakpoints [0,0], [81,50], [210,162] and [255,92]. 

c. The B colormap is a piecewise linear map specified by the following 
breakpoints [0,255], [120,225], [178,251] and [255,255]. 

d. The a colormap is a piecewise linear map specified by the following 
breakpomts [0,255], [120,225], [178,251] and [255,255]. 

e. Convert the floating point Ip image to an 8-bit image, in the range [0,255] 
by rounding the product of each Ip image pixel by 255 . 

f. Use the tissue colormap to get RGBa pixel values for the disease display layer. 
[0686] Figure 116 shows tiie color transformation used in overlaying Ike disease display layer 
onto the reference image, as in the overlay image 2360 of Figure 1 14B. The first colorbar 2374 
in Figure 1 1 6 shows the blended colors from yellow to blue that correspond to values of disease 
probability pc23(^)> depicted on the x-axis 2375. A color corresponding to the average tissue 
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color is detennined, as shown in coloibar 2378. The average tissue color is blended into the 
probability-correlated yellow-to-blue colorbar 2374 so that tiie yellow color is blended into the 
reference image where the disease probabilily, as indicated by the filtered disease display layer, 
is low. TWs avoids a sharp discontinuity between the disease map and the tissue. In one 

5 embodiment, the disease display layer and the base (reference) image are combined by using 
alpha-channel blending, where the alpha channel is as shown in step #4 of the above method to 
create a disease display layer. The disease display layer is overlaid upon the base image with 
blending controlled by the computed alpha channel values according to Equation 143 as foUows: 
(Overlay Image Pixel) = a-(Disease Display Layer Pixel) + (l-a)<Base Image Pixel) (143) 

10 [0687] Next, the disease probability display metiiod 138 of Figure 1 includes determining an 
"hideterminate" mask from the classification results in step 1432 of Figure 74, where 
indeterminate regions are indicated by a gmy "see-through'* Crosshatch pattern. For an 
exemplary reference image, interrogation points classified as "Indeterminate" in step 1432 of 
Figure 74 indicate where the indeterminate mask is activated. The indeterminate Crosshatch 

15 mask is then combined with the output overlay image, as is shown in the overlay image 2360 of 
Figure 114B. Here, indeterminate regions 2366 are indicated in shadowed regions around the 
edge of the tissue sample. 

[0688] hi one embodhnent, the disease probability display method 138 of Figure 1 mcludes 
creating an indeterminate ciosshatch mask according to the following steps: 
20 1 .Create image, msk, of base image size and set to 0. 

2. Draw disks of radius 0.75 mm centered at the coordinate of each indeterminate 

interrogation point 

3. Erode mask image 3 times to obtain erodMsk = erod (msk, 3). 

4. Compute image bmary perimeter, perMsk, of erodWIsk: 

25 perMsk not (erod (orodMsk, 1)) AND (erodMsk)), 1). 

5. Compute indeterminate Crosshatch mask: 

a. Retrieve Crosshatch image, xhatch, defined by a horizontal pitch of 1 0 
pixels, a vertical pitch of 20 pixels, a Crosshatch slope of 2 and a grey 
value of (166,166,166). 
30 b. Perform logical OR oferodMsk and xhatch to obtain xhatchMsk. 

c. Perform logical OR of xhatchMsk with perMsk. 
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{06891 Next, the disease probability display method 138 of Figure 1 includes deteimining a 
'•necrosis" mask fiom the classification results in step 1432 of Figure 74, where necrotic regions 
are indicated by a green "see-tiirough" trellis pattern. Figure 1 17A depicts an exemplary 
reference image 2388 of cervical tissue having necrotic regions. For an exemplary refwence 
5 image, interrogation points classified as "Necrotic" in step 1432 of Figure 74 indicate where the 
"necrosis" mask is activated. A necrosis trellis mask is included in the overlay image, as is 
shown in the overlay image 2396 of Figure 1 17B. 

[0690] In one embodiment, the disease probability display method 138 of Figure 1 includes 
creating a necrosis trellis mask accordii^ to &e following steps: 
10 1 .Create image, msk, of base image size, and set it to 0. 

2.Draw disks of radius 0.75 mm centered at the coordinate of each necrotic tissue 

interrogation point 
S.Erode mask image 3 times to obtam erodMsk = eiod (msk, 3). 

4. Compute unage binary perimeter, perMsk, of erodMsk: 

15 perMsk = not (erod (erodMsk,!)) AND (erodMsk)), 1). 

5. Compute necrotic tissue trellis mask: 

a. Retrieve trellis image, trellis, defmed by a horizontal pitch of 8 pixels, a 
vertical pitch of 8 pixels, a Ime thickness of 2 and a green value of 
(0,255,104). 

20 b. Perform logical OR of erodMsk and xhatch to obtain trelHsMsk. 

c. Perform logical OR of trellfsMsk witii perMsk. 

[06911 The result of the disease probability display method 138 of Figure 74 is a state-of- 
healfh *'map" of a tissue sample, with annotations mdicating indeterminate re^ons, neootic 
regions, and/or regions of low-to-high probability of high-grade disease. The disease display 

25 overlay images contain indeterminate regions and regions of low-to-high probability of CDST 2/3 . 
[0692] In one embodiment, the disease display overlay unage is produced immediately 
following a patient scan in which spectral and image data are acqmred and processed. This 
allows a physicial to provide on-tiie-spot diagnostic review immediately followii^ the scan. 
[0693] Figure 118 is a block diagram of an instrument 2400 tiiat employs the tissue 

30 characterization system of the invention, additionally fitted with a tool 2402 for markmg suspect 
portions of tissue and a tool 2404 for treating tiie suspect portions. The ti«atment tool 2404 can 
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be used for excising tissue for biopsy, for treating tissue, for applying a chemical agent, and/or 
for ablating tissue. The treatment tool 2404 and/or the marking tool 2402 may be integral to the 
console 140 or the probe 142, or they may be separately provided. Furthermore, a colposcope 
2406 may be provided as integral to the probe 142, or may be separately provided as part of a 
5 system of the invention. For example, a system of the invention may include a probe 142 with 
an integral colposcopic functionality that allows obtaining a reference image of the tissue on 
vMch an overlay map can be superimposed. Alternatively, the colposcope may be a separate 
part of the system. The superposed map may be viewed on a display surface, or may be viewed 
through a viewfinder, for example. 

10 

EQUIVALENTS 

[0694] While the invention has been particularly shown and described with reference to 
specific preferred embodiments, it should be understood by those skilled in the art that various 
changes in form and detail may be made therein without departing from the spirit and scope of 
15 the invention as defined by the appended claims. 
What is claimed is: 
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CLAIMS 

1 1 . A diagnostic method comprising the steps of: 

2 (a) examining an in-vivo tissue sample using an optical signal detection system adapted to 

3 automatically assign a classification to each of a plurality of regions of the tissue sample; 

4 (b) creating an overlay map visually indicating the classifications assigned to the regions of 

5 the tissue sample; and 

6 (c) displaying the overlay map to facilitate identification of suspect portions of the tissue 

7 sample. 

1 2. The method of claim 1 , wherein step (c) comprises superimposing the overlay map onto 

2 an image of the tissue sample. 

1 3 . The method of claim 2, wherein the image of the tissue sample is a reference image. 

1 4. The method of claim 3, wherein the tissue sample comprises cervical tissue and the 

2 reference image is a colposcopic image. 

1 5. The method of claim 2, wherein step (c) comprises displaying the overlay map 

2 superimposed onto a real-time colposcopic ims^e of the tissue sample- 

1 6. The method of claim 1 , wherem step (c) comprises projecting the overlay map onto the 

2 tissue sample. 

1 7. The method of claim 1, wherem steps (a) through (c) are performed during the course of 

2 a single patient visit 

1 • 8. The method of claim 1, wherein the displaying step is performed substantially 

2 contemporaneously with the examining step. 
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1 9. The method of claim 1 , wherein the classification in step (a) comprises a tissue-class 

2 probability. 

1 10. The method of claun 1 , wherein the classifications assigned to the regions of the tissue 

2 sample comprise at least one of the categories selected &om the group consisting of necrotic, 

3 CIN 2/3, NED, and indeterminate. 

1 11, The method of claim 1, fiirther comprising the steps of: 

2 (d) identifying at least one suspect portion of the tissue sample; and 

3 (e) marking the at least one suspect portioa 

1 12. The method of claim 1 1, wherein step (e) comprises marking the at least one suspect 

2 portion using at least one of an endogenous agent and an exogenous agent 

1 13. The method of claim 1 1, wherein step (e) comprises marking at least one suspect portion 

2 using at least one of a photobleaching technique and a photoactivation technique. 

1 1 4. The method of claim 1 1 , wherein step (e) comprises marking tiie at least one suspect 

2 region for follow-up examination. 

1 15. The method of claim 1 1 , wherein step (e) comprises marking the at least one suspect 

2 region for treatment 

1 16. The method of claim 1 1, wherein steps (a) through (e) are performed durmg a single 

2 patient visit. 

1 17. The method of claim 1 , further comprising the steps of: 

2 (d) identifying at least one suspect portion of the tissue sample; and 

3 (e) excising tissue &om the at least one suspect portion for biopsy. 
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1 18. The method of claim 1 , further comprising fbe steps of: 

2 (d) identifying at least one suspect portion oftiie tissue sample; and 

3 (e) treating the at least one suspect portion. 

1 19. The metiiod of claim 1 8, wherein step (e) comprises performing at least one of: 

2 photodynamic therapy, cryotherapy, and direct chemical treatment. 

1 20. The method of claim 1 8, wherein step (e) comprises performing photodynamic theKq)y 

2 using at least one photosensitive agent. 

1 21. The method of claim 20, wherem the at least one photosensitive agent comprises at least 

2 one of an exogenous agent and an endogenous agent 

1 22. The method of claim 20, whweiu flie at least one photosensitive agent comprises at least 

2 one of a hematopoiphyrin, a phAalocyanine, and a chlorin. 

1 23. The method ofdaim 20, wherein the at least one photosensitive agent comprises at least 

2 one of dihematoporphyrin, 5-aminolevulinic acid, protoporphyrin IX, temoporfin, and meso- 

3 tetrahydroxyphenylchlorin. 

1 24. , The method ofclaim 18, wherein step (e) comprises removing tissue from the at least one 

2 suspect portion using laser ablation. 

1 25. The method of claim 1 8, wherein steps (a) through (e) are performed during a single 

2 patient visit. 

1 26. A system for detection of suspect portions of a tissue sample, the system comprising: 

2 (a) an optical signal detection apparatus adapted to obtain at least one optical signal from 

3 each of a plurality of regions of a tissue sample; 
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4 (b) a memory tiiat stores code defining a set of instructions; 

5 (c) a processor that executes the instroctions thereby to: 

6 (i) identify a characteristic ofeachofthe plurality ofregions based at least in part on 

7 the at least one optical signal; and 

8 (ii) define an overlay map for visually indicating the characteristics identified in step 

9 (i); and 

10 (d) a display for facilitating identification of suspect portions of Ihe tissue sample according 

11 to the overlay map. 

1 27. The system ofclaim 26, wherein the optical signal detection apparatus comprises 

2 illuminating optics for iUuminating each of the plurality ofregions of the tissue sample and 

3 collecting optics for receiving at least one optical signal from each of the plurality of regions. 

1 28. The system ofclaim 27, wherein the iUuniinating optics comprise a v\*ite light source 

2 and an ultraviolet light source and wherein the at least one optical signal comprises a 

3 fluorescence spectrum and at least one reflectance spectrum. 

1 29. The system ofclaim 28, wherein the at least one optical signal comprises a fluorescence 

2 spectrum and two reflectance spectra. 

1 30. The system of claim 27, wherein the optical signal detection apparatus comprises optics 

2 for obtaining a sequence of images of the tissue sample useful for motion compensation. 

1 31. The system of claim 27, wherein each of the plurality ofregions has a dimension of about 

2 a millimeter. 

1 32. The system ofclaim 31, wherein each ofthe plurality ofregions has a diameter of about 

2 0.7 mm. 
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1 33. The system of claim 27, wherein the optical signal detection apparatus is adapted to 

2 complete a cervical scan in less Him about 30 seconds. 

1 34. The system of claim 33, wherein the optical signal detection apparatus is adapted to 

2 complete a cervical scan in less than about 1 5 seconds. 

1 35. The system of claim 26, wherein the display shows the overlay map superimposed onto 

2 an image of the tissue sample. 

1 36. The system of claim 35, wherein the image of the tissue sanq)le is a reference image. 

1 37. The system of claim 35, wherein the tissue sample comprises cervical tissue and the 

2 reference image is a colposcopic image. 

1 38. The system of claim 35, wherein the display shows the overlay map superimposed onto a 

2 real-time colposcopic image of the tissue sample. 

1 39. The system of claim 35, wherein the display can be viewed through a viewfinder. 

1 40. The system of claim 3 5, wherein the display is a surface upon which the overlay map can 

2 be projected. 

1 41 . The system of claim 26, wherein the display is a projection of the overlay map onto the 

2 tissue sample. 

1 42. The system of claim 26, further comprising a tool for marking a suspect region of the 

2 tissue sample. 

1 43. The system of claim 26, further comprising an instrument for excising tissue from a 

2 suspect portion ofthe tissue sample for biopsy. 
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1 44. The system of claim 26, further comprising an instrument for treating tissue from a 

2 suspect portion of the tissue sample. 

1 45. The system of claim 26, further comprising a laser for ablating tissue within a suspect 

2 portion of the tissue sample. 

1 46. The system of claim 26, further comprising a light source for performing photodynanuc 

2 therapy to treat the suspect portions of the tissue sample. 

1 47. An optical signal detection apparatus for performing a cervical scan, the cqpparatus 

2 comprismg: 

3 (a) illuminating optics for sequentially illuminating each of a plurality of regions of a tissue 

4 sample, wherein the illimiinating optics comprise a white light source and an ultraviolet light 

5 source; and 

6 (b) collecting optics for receiving at least one optical signal from each of the plurality of 

7 regions, wherein the at least one optical signal comprises a fluorescence spectrum and at least 

8 one reflectance spectrum, wherein each of the plurality of regions has a dimension of about a 

9 millimeter, and wherein the optical signal detection apparatus is adapted to complete a cervical 

10 scan in less than about 30 seconds. 

1 48. The apparatus of claim 47, wherein the optical signal detection apparatus is adapted to 

2 complete a cervical scan in less than about 15 seconds. 

1 49. The apparatus of claim 47, wherein the plurality of regions of the tissue sample 

2 comprises from about 100 to about 1000 regions of the tissue sample. 

1 50. The apparatus of claim 49, wherein the pluralitJ^ of regions of the tissue sample 

2 comprises about 500 regions. 
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1 5 1 • The apparatus of claim 47, comprising a light source for performing photodynamic 

2 therapy. 

1 52, A method for idmtifying a characteristic of each of a plurality of regions of a tissue 

2 sample, the method comprising the steps of: 

3 (a) obtaining at least one optical signal from each of a plurality of regions of a tissue sample 

4 following application of a contrast agent and at least one of an optical probe and a biological- 

5 responsive probe to the tissue sample; and 

6 (b) automatically identifying a characteristic of each of the plurality of regions based at least 

7 in part on the at least one optical signal. 

1 53. The method of claim 52, wherein the optical probe comprises a spectroscopic enhancer. 

1 54. The method of claun 52, wherein the biological-responsive probe comprises telomerase. 

1 55. The method of claim 52, wherein step (b) comprises applying at least one of an optical 

2 probe and a biological-responsive probe to the tissue sample in order to detect at least one 

3 member selected from the group consisting of collagen, porphyrin, FAD, and NADH. 

1 56. The method of clmm 52, frirther comprising the steps of: 

2 (e) creating an overlay map visually indicating the characteristics identified in step (d); and 

3 (f) displaying the overlay map to facilitate identification of suspect portions of the tissue 

4 sample. 

1 57. The method of claim 52, wherem the at least one optical signal comprises a fluorescence 

2 spectrum and at least one reflectance spectrum, and wherein step (b) comprises identifying a 

3 characteristic of a subset of the plurality of regions based at least in part on a fluorescent 

4 biomarker signal. 
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499 Interrogation Points 
Selection Diameter: 25.8000 mm 
Spot Size: 0.7000 mm 
Pitoli: 1.1000 mm 
Nominal Diameter: 25.000 mm 
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(57) Abstract: The invention provides 
systems (102) for automatically localizing 
areas of disease within a tissue sample 
using fluorescence spectra, reflectance 
spectra, and video images (102) obtained 
from the sample. The invention further 
provides systems for displaying (138), 
marking, and treating the identified areas 
so that healthy surrounding tissue suffers 
minimal impact upon treatment and/or 
excision of the diseased tissue (104). The 
invention facilitates minimally-invasive 
treatment by accurately directing treatment 
to only those areas that are identified as 
diseased. 
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